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Abstract 


A  logical  framework  is  a  general  meta-language  for  specifying  and  implementing  de¬ 
ductive  systems,  given  by  axioms  and  inference  rules.  Based  on  a  higher-order  logic 
programming  interpretation,  it  supports  executing  logical  systems  and  reasoning  with 
and  about  them,  thereby  reducing  the  effort  required  for  each  particular  logical  system. 

In  this  thesis,  we  describe  different  techniques  to  improve  the  overall  performance 
and  the  expressive  power  of  higher-order  logic  programming.  First,  we  introduce  tabled 
higher-order  logic  programming,  a  novel  execution  model  where  some  redundant  infor¬ 
mation  is  eliminated  using  selective  memoization.  This  extends  tabled  computation 
to  the  higher-order  setting  and  forms  the  basis  of  the  tabled  higher-order  logic  pro¬ 
gramming  interpreter.  Second,  we  present  efficient  data-structures  and  algorithms  for 
higher-order  proof  search.  In  particular,  we  describe  a  higher-order  assignment  algo¬ 
rithm  which  eliminates  many  unnecessary  occurs  checks  and  develop  higher-order  term 
indexing.  These  optimizations  are  crucial  to  make  tabled  higher-order  logic  program¬ 
ming  successful  in  practice.  Finally,  we  use  tabled  proof  search  in  the  meta-theorem 
prover  to  reason  efficiently  with  and  about  deductive  systems.  It  takes  full  advantage 
of  higher-order  assignment  and  higher-order  term  indexing. 

As  experimental  results  demonstrate,  these  optimizations  taken  together  constitute 
a  significant  step  toward  exploring  the  full  potential  of  logical  frameworks  in  practice. 
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Chapter  1 
Introduction 


A  logical  framework  is  a  general  meta-language  for  specifying  and  implementing  de¬ 
ductive  systems,  given  by  axioms  and  inference  rules.  Examples  of  deductive  systems 
are  plentiful  in  computer  science.  In  computer  security,  we  find  authentication  and 
security  logics  to  describe  access  and  security  criteria.  In  programming  languages,  we 
use  deductive  systems  to  specify  the  operational  semantics,  type-systems  or  other  as¬ 
pects  of  the  run-time  behavior  of  programs.  Recently,  one  major  application  of  logical 
frameworks  has  been  in  the  area  of  “certified  code” .  To  provide  guarantees  about  the 
behavior  of  mobile  code,  safety  properties  are  expressed  as  deductive  systems.  The 
code  producer  then  verifies  the  program  according  to  some  predetermined  safety  pol¬ 
icy,  and  supplies  a  binary  executable  together  with  its  safety  proof  (certificate).  Before 
executing  the  program,  the  host  machine  then  quickly  checks  the  code’s  safety  proof 
against  the  binary.  The  safety  policy  and  the  safety  proofs  can  be  expressed  in  the 
logical  framework  thereby  providing  a  general  safety  infrastructure. 

There  are  two  main  variants  of  logical  frameworks  which  are  specifically  designed 
to  support  the  implementation  of  deductive  systems.  AProlog  and  Isabelle  are  based 
on  hereditary  Harrop  formulas,  while  the  Twclf  system  [53]  is  an  implementation  of 
the  logical  framework  LF,  a  dependency  typed  lambda  calculus.  In  this  thesis,  we 
will  mainly  focus  on  the  latter.  By  assigning  a  logic  programming  interpretation  to 
types  [47],  we  obtain  a  higher-order  logic  programming  language.  Higher-order  logic 
programming  in  Twelf  extends  traditional  first-order  logic  programming  in  three  ways: 
First,  we  have  a  rich  type  system  based  on  dependent  types,  which  allows  the  user  to 
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define  her  own  higher-order  data-types  and  supports  higher-order  abstract  syntax  [52] . 
Variables  in  the  object  language  can  be  directly  represented  as  variables  in  the  meta¬ 
language  thereby  directly  inheriting  capture-avoiding  substitution  and  bound  variable 
renaming.  Second,  we  not  only  have  a  static  set  of  program  clauses,  but  clauses  may 
be  introduced  dynamically  and  used  within  a  certain  scope  during  proof  search.  Third, 
we  have  an  explicit  notion  of  proof,  i.e.,  the  logic  programming  interpreter  does  not 
only  return  an  answer  substitution  for  the  free  variables  in  the  query,  but  also  the 
actual  proof  of  the  query  as  a  term  in  the  dependency  typed  lambda-calculus.  This 
stands  in  sharp  contrast  to  higher-order  features  supported  in  many  traditional  logic 
programming  languages  (see  for  example  [13])  where  we  can  encapsulate  predicate 
expressions  within  terms  to  later  retrieve  and  invoke  such  stored  predicates.  Twelf’s 
higher-order  logic  programming  interpreter  is  complemented  by  a  meta-theorem  prover, 
which  combines  generic  proof  search  based  on  higher-order  logic  programming  with 
inductive  reasoning  [53,  63]. 

The  Twclf  system  has  been  successfully  used  to  implement,  execute  and  reason 
about  a  wide  variety  of  deductive  systems.  However,  experience  with  real-world  appli¬ 
cations  in  different  projects  on  certified  code  [4,  15,  3]  have  increasingly  demonstrated 
the  limitations  of  Twelf’s  higher-order  logic  programming  proof  search.  To  illustrate, 
let  us  briefly  consider  the  foundational  proof-carrying  code  project  at  Princeton.  As 
part  of  this  project,  the  researchers  at  Princeton  have  implemented  between  70,000 
and  100,000  lines  of  Twelf  code,  which  includes  data-type  definitions  and  proofs.  The 
higher-order  logic  program,  which  is  used  to  execute  safety  policies,  consists  of  over 
5,000  lines  of  code,  and  over  600  -  700  clauses.  Such  large  specifications  have  put  to 
test  implementations  of  logical  frameworks  and  exposed  several  problems.  First,  per¬ 
formance  of  the  higher-order  logic  programming  interpreter  may  be  severely  hampered 
by  redundant  computation,  leading  to  long  response  times  and  slow  development  of 
formal  specifications.  Second,  many  straightforward  specifications  of  formal  systems, 
for  example  recognizers  and  parsers  for  grammars,  rewriting  systems,  type  systems 
with  subtyping  or  polymorphism,  are  not  executable,  thus  requiring  more  complex 
and  sometimes  less  efficient  implementations.  Thirdly,  redundancy  severely  hampers 
the  reasoning  with  and  about  deductive  systems  in  general,  limiting  the  use  of  the 
meta-theorem  prover. 

In  applications  to  certified  code,  efficient  proof  search  techniques  not  only  play  an 
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important  role  to  execute  safety  polices  and  generate  a  certificate  that  a  given  program 
fulfills  a  specified  safety  policy,  but  it  also  can  be  used  to  check  the  correctness  of  a 
certificate  [42],  Necula  and  Rahul  [42]  propose  as  a  certificate  a  bit-string  of  the 
non-deterministic  choices  in  the  proof.  Hence,  a  proof  can  be  checked  by  guiding  the 
higher-order  logic  programming  interpreter  with  the  bit-string  and  reconstructing  the 
actual  proof.  As  pointed  out  by  Necula  and  Lee,  typical  safety  proof  in  the  context 
of  certified  code  commonly  have  repeated  sub-proofs  that  should  be  hoisted  out  and 
proved  only  once.  The  replication  of  common  sub-proofs  leads  to  redundancy  in  the 
bit-strings  representing  the  safety  proof  and  it  may  take  longer  to  reconstruct  the  safety 
proof  using  a  guided  higher-order  logic  programming  interpreter. 

In  this  thesis,  we  develop  different  techniques  which  improve  the  overall  performance 
and  the  expressive  power  of  the  higher-order  logic  programming  interpreter.  We  also 
apply  these  ideas  in  the  meta-theorem  prover  to  overcome  existing  limitations  when 
reasoning  about  deductive  systems.  These  optimizations  taken  together  constitute  a 
significant  step  toward  exploring  the  full  potential  of  logical  frameworks  in  real-world 
applications.  Some  of  the  work  in  this  thesis  has  been  previously  published  in  different 
forms  [54,  55,  56,  57,  41] 


Contributions 

The  contributions  in  this  thesis  are  in  three  main  areas:  First,  we  introduce  tabled 
higher-order  logic  programming,  a  novel  execution  model  where  some  redundant  in¬ 
formation  is  eliminated  using  selective  memoization.  This  forms  the  basis  of  the 
tabled  higher-order  logic  programming  interpreter.  Second,  we  develop  efficient  data- 
structures  and  algorithms  for  higher-order  proof  search.  These  optimizations  are  cru¬ 
cial  to  make  tabled  higher-order  logic  programming  successful  in  practice.  Although 
we  develop  these  techniques  in  the  context  of  tabled  logic  programming,  they  are 
also  independently  useful  and  important  to  other  areas  such  as  higher-order  rewrit¬ 
ing,  higher-order  theorem  proving  and  higher-order  proof  checking.  Third,  we  use 
memoization-based  proof  search  in  the  meta-theorem  prover,  to  reason  efficiently  with 
and  about  deductive  systems.  This  demonstrates  the  importance  of  memoization  in 
general.  Next,  we  will  discuss  briefly  each  of  these  contributions. 
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Tabled  higher-order  logic  programming 

Tabled  first-order  logic  programming  has  been  successfully  applied  to  solve  complex 
problems  such  as  implementing  recognizers  and  parsers  for  grammars  [68] ,  representing 
transition  systems  CCS  and  writing  model  checkers  [16].  The  idea  behind  it  is  to  elim¬ 
inate  redundant  computation  by  memoizing  sub-computation  and  re-using  its  results 
later.  The  resulting  search  procedure  is  complete  and  terminates  for  programs  with 
the  bounded-term  size  property.  The  XSB  system  [62],  a  tabled  logic  programming 
system,  demonstrates  impressively  that  tabled  together  with  non-tabled  programs  can 
be  executed  efficiently  in  the  first-order  setting. 

The  success  of  memoization  in  first-order  logic  programming  strongly  suggests  that 
memoization  may  also  be  valuable  in  higher-order  logic  programming.  In  fact,  Necula 
and  Lee  point  out  in  [44]  that  typical  safety  proofs  in  the  context  of  certified  code 
commonly  have  repeated  sub-proofs  that  should  be  hoisted  out  and  proved  only  once. 
Memoization  has  potentially  three  advantages.  First,  proof  search  is  faster  thereby 
substantially  reducing  the  response  time  to  the  programmer.  Second,  the  proofs  them¬ 
selves  are  more  compact  and  smaller.  This  is  especially  important  in  applications  to 
secure  mobile  code  where  a  proof  is  attached  to  a  program,  as  smaller  proofs  take  up  less 
time  to  check  and  transmit  to  another  host.  Third,  substantially  more  specifications, 
for  example  recognizers  and  parser  for  grammars,  evaluators  based  on  rewriting  or  type 
systems  with  subtyping,  are  executable  under  the  new  paradigm  thereby  extending  the 
power  of  the  existing  system. 

Using  memoization  in  higher-order  logic  programming  poses  several  challenges, 
since  we  have  to  handle  type  dependencies  and  may  have  dynamic  assumptions  which 
are  introduced  during  proof  search.  This  is  unlike  tabling  in  XSB,  where  we  have  no 
types  and  it  suffices  to  memoize  atomic  goals.  Moreover,  most  descriptions  of  tabling 
in  the  first-order  setting  are  closely  oriented  on  the  WAM  (Warren  Abstract  Machine) 
making  it  hard  to  transfer  tabling  techniques  and  design  extensions  to  other  logic 
programming  interpreters. 

In  this  thesis,  we  introduce  a  novel  execution  model  for  logical  frameworks  based 
on  selective  memoization. 

Proof-theoretic  characterization  of  uniform  proofs  and  memoization  We  give 
a  proof-theoretic  characterization  of  tabling  based  on  uniform  proofs,  and  show 
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soundness  of  the  resulting  interpreter.  This  provides  a  high-level  description  of  a 
tabled  logic  programming  interpreter  and  separates  logical  issues  from  procedural 
ones  leaving  maximum  freedom  to  choose  particular  control  mechanisms. 

Implementation  of  a  tabled  higher-order  logic  programming  interpreter  We 

give  a  high-level  description  of  a  semi-functional  implementation  for  adding  tabling 
to  a  higher-order  logic  programming  interpreter.  We  give  an  operational  interpre¬ 
tation  of  the  uniform  proof  system  and  discuss  some  of  the  implementation  issues 
such  as  suspending  and  resuming  computation,  retrieving  answers,  and  trailing. 
Unlike  other  description,  it  does  not  require  an  understanding  or  modifications, 
and  extensions  to  the  WAM  (Warren  abstract  machine).  It  is  intended  as  a  high- 
level  explanation  and  guide  for  adding  tabling  to  an  existing  logic  programming 
interpreter.  This  is  essential  for  rapidly  prototyping  tabled  logic  programming 
interpreters,  even  for  linear  logic  programming  and  other  higher-order  logic  pro¬ 
gramming  systems. 

Case  studies  We  discuss  two  case  studies  to  illustrate  the  use  of  memoization  in  the 
higher-order  setting.  We  consider  a  parser  and  recognizer  for  first-order  formulas 
into  higher-order  abstract  syntax.  To  model  left  and  right  associativity  of  the 
different  connectives,  we  mix  left  and  right  recursion  in  the  specification  of  the 
parser.  Although  this  closely  models  the  grammar,  it  leads  to  an  implementation 
which  is  not  executable  with  traditional  logic  programming  interpreters  which 
are  based  on  depth-first  search. 

The  second  case  study  is  an  implementation  of  a  bi-directional  type-checker  by 
Davies  and  Pfenning  [17].  The  type-checker  is  executable  with  the  original  logic 
programming  interpreter,  which  performs  a  depth-first  search.  However,  redun¬ 
dant  computation  may  severely  hamper  its  performance  as  there  are  several 
derivations  for  proving  that  a  program  has  a  specified  type. 

Efficient  data-structures  and  algorithms 

Efficient  data-structures  and  implementation  techniques  play  a  crucial  role  in  utilizing 

the  full  potential  of  a  reasoning  environment  in  large  scale  applications.  Although 
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this  need  has  been  widely  recognized  for  first-order  languages,  efficient  algorithms  for 
higher-order  languages  are  still  a  central  open  problem. 

Proof-theoretic  foundation  for  existential  variables  based  on  modal  logic 

We  give  a  dependent  modal  lambda  calculus,  which  extends  the  theory  of  the 
logical  framework  LF  [29]  conservatively  with  modal  variables.  Modal  variables 
can  be  interpreted  as  existential  variables,  thereby  clearly  distinguishing  them 
from  ordinary  bound  variables.  This  is  critical  to  achieve  a  simplified  account  of 
higher-order  unification  and  allows  us  to  justify  different  optimizations  such  as 
as  lowering,  raising,  and  linearization  [57,  41].  It  also  serves  as  a  foundation  for 
designing  higher-order  term  indexing  strategies. 

Optimizing  unification  Unification  lies  at  the  heart  of  logic  programming,  theorem 
proving,  and  rewriting  systems.  Thus,  its  performance  affects  in  a  crucial  way 
the  global  efficiency  of  each  of  these  applications.  Higher-order  unification  is 
in  general  undecidablc,  but  decidable  fragments,  such  as  higher-order  patterns 
unification,  exist.  Unfortunately,  the  complexity  of  this  algorithm  is  still  at  best 
linear,  which  is  impractical  for  any  useful  programming  language  or  practical 
framework.  In  this  thesis,  we  present  an  assignment  algorithm  for  linear  higher- 
order  patterns  which  factors  out  unnecessary  occurs  checks.  Experiments  show 
that  we  get  a  speed-up  by  up  to  a  factor  2-5  making  the  execution  of  some 
examples  feasible.  This  is  a  significant  step  toward  efficient  implementation  of 
higher-order  reasoning  systems  in  general  [57]. 

Higher-order  term  indexing  Proof  search  strategies,  such  as  memoization,  can  only 
be  practical  if  we  can  access  the  memo-table  efficiently.  Otherwise,  the  rate  of 
drawing  new  conclusions  may  degrade  sharply  both  with  time  and  with  an  in¬ 
crease  of  the  size  of  the  memo-table.  Term  indexing  aims  at  overcoming  pro¬ 
gram  degradation  by  sharing  common  structure  and  factoring  common  opera¬ 
tions.  Higher-order  term  indexing  has  been  a  central  open  problem,  limiting 
the  application  and  the  potential  impact  of  higher-order  reasoning  systems.  In 
this  thesis,  we  develop  and  implemented  higher-order  term  indexing  techniques. 
They  improve  performance  by  up  to  a  factor  of  9,  illustrating  the  importance  of 
indexing  [56]. 
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Meta-theorem  proving  based  on  memoization 

The  traditional  approach  for  supporting  theorem  proving  in  logical  frameworks  is  to 
guide  proof  search  using  tactics  and  tacticals.  Tactics  transform  a  proof  structure 
with  some  unproven  leaves  into  another.  Tacticals  combine  tactics  to  perform  more 
complex  steps  in  the  proof.  Tactics  and  tacticals  are  written  in  ML  or  some  other 
strategy  language.  To  reason  efficiently  about  some  specification,  the  user  implements 
specific  tactics  to  guide  the  search.  This  means  that  tactics  have  to  be  rewritten  for 
different  specifications.  Moreover,  the  user  has  to  understand  how  to  guide  the  prover 
to  find  the  proof,  which  often  requires  expert  knowledge  about  the  systems.  Proving 
the  correctness  of  the  tactic  is  itself  a  complex  theorem  proving  problem. 

The  approach  taken  in  the  Twelf  system  is  to  endow  the  framework  with  the  oper¬ 
ational  semantics  of  logic  programming  and  design  general  proof  search  strategies  for 
it.  Twelf’s  meta-theorem  prover  combines  general  proof  search  based  on  higher-order 
logic  programming  with  inductive  reasoning.  Using  the  proof-theoretic  characteriza¬ 
tion  of  tabling,  we  develop  a  general  memoization-based  proof  search  strategy  which 
is  incorporated  in  Twclf’s  meta-theorem  prover.  As  experiments  demonstrate,  elimi¬ 
nating  redundancy  in  meta-theorem  proving  is  critical  to  prove  properties  about  larger 
and  more  complex  specifications.  We  discuss  several  examples  including  type  preserva¬ 
tion  proofs  for  type-system  with  subtyping,  several  inversion  lemmas  about  refinement 
types,  and  reasoning  in  classical  natural  deduction.  These  examples  include  several 
lemmas  and  theorems  which  were  not  previously  provable.  Moreover,  we  show  that  in 
many  cases  no  bound  is  needed  on  memoization-based  search.  As  a  consequence,  if 
a  sub-case  is  not  provable,  the  user  knows,  that  in  fact  no  proof  exists.  This  in  turn 
helps  the  user  to  revise  the  formulation  of  the  theorem  or  the  specification.  Overall 
the  benefits  of  memoization  are  an  important  step  towards  a  more  robust  and  more 
efficient  meta-theorem  prover. 
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Chapter  2 


Dependently  typed  lambda  calculus 
based  on  modal  type  theory 


In  this  chapter,  we  introduce  a  dependently  typed  lambda  calculus  based  on  modal 
type  theory.  The  underlying  motivation  for  this  work  is  to  provide  a  logical  founda¬ 
tion  for  the  implementation  of  logical  frameworks  and  the  design  choices  one  has  to 
make  in  practice.  One  such  choice  is  for  example  the  treatment  of  existential  variables. 
Existential  variables  are  usually  implemented  via  mutation.  However,  previous  formu¬ 
lations  of  logical  frameworks  [28,  29]  do  not  capture  or  explain  this  technique.  The 
framework  presented  in  this  chapter  serves  as  a  foundation  for  the  subsequent  chapters 
on  higher-order  unification,  higher-order  proof  search,  and  higher-order  term  indexing. 

We  present  here  an  abstract  view  of  existential  variables  based  on  modal  type 
theory.  Existential  variables  u  (or  v)  are  treated  as  modal  variables.  This  allows  us 
to  reason  about  existential  variables  as  first-class  objects  and  directly  explain  many 
optimizations  done  in  implementations.  In  this  chapter,  we  will  conservatively  extend 
the  theory  of  the  logical  framework  LF  given  by  Harper  and  Pfenning  in  [29]  with 
modal  variables.  Following  Harper  and  Pfenning,  we  will  introduce  this  language  and 
show  that  definitional  equality  remains  decidable.  In  addition,  normalization  and  type¬ 
checking  are  decidable. 
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2.1  Motivation 

Before  presenting  the  foundation  for  dependently  typed  existential  variables,  we  briefly 
motivate  our  approach  based  on  modal  logic.  Following  the  methodology  of  Pfenning 
and  Davies  [51],  we  can  assign  constructive  explanations  to  modal  operators.  A  key 
characteristic  of  this  view  is  to  distinguish  between  propositions  that  are  true  and 
propositions  that  are  valid.  A  proposition  is  valid  if  its  truth  does  not  depend  on  the 
truth  of  any  other  propositions.  This  leads  to  the  basic  hypothetical  judgment 

Ai  valid , . . .  An  valid ;  B\  true , . . . ,  Bm  true  b  C  true. 

Under  the  multiple-world  interpretation  of  modal  logic,  C  valid  corresponds  to 
C  true  in  all  reachable  worlds.  This  means  C  true  without  any  assumptions,  except 
those  that  are  assumed  to  be  true  in  all  worlds.  We  can  generalize  this  idea  to  also 
capture  truth  relative  to  a  set  of  specified  assumptions  by  writing  C  valid  T,  where  T 
is  the  abbreviation  for  Cj  true, . . . ,  Cn  true.  In  terms  of  the  multiple  world  semantics, 
this  means  that  C  is  true  in  any  world  where  C\  through  Cn  are  all  true  and  we  say  C 
is  valid  relative  to  the  assumptions  in  T.  Hypotheses  about  relative  validity  are  more 
complex  now,  so  our  general  judgment  form  is 

Ai  valid  Ti, . . . ,  An  valid  B i  true, . . . ,  Bm  true  b  C  true 

While  it  is  interesting  to  investigate  this  modal  logic  above  in  its  own  right,  it  does 
not  come  alive  until  we  introduce  proof  terms.  In  this  chapter,  we  investigate  the 
use  of  a  modal  proof  term  calculus  as  a  foundation  for  existential  variables.  We  will 
view  existential  variables  u  as  modal  variables  of  type  A  in  a  context  T  while  bound 
variables  are  treated  as  ordinary  variables.  This  allows  us  to  distinguish  between 
existential  variables  ■u::(Ti-A)  for  relative  validity  assumptions  A  valid  T  declared  in  a 
modal  context,  and  x:A  for  ordinary  truth  assumptions  A  true  declared  in  an  (ordinary) 
context.  If  we  have  an  assumption  A  valid  T  we  can  only  conclude  A  true  if  we  can 
verify  all  assumptions  in  T. 

A,  A  valid  T,  A';  T  b  T 
A,  A  valid  T,  A';  T  b  A  true 

In  other  words,  if  we  know  A  true  in  T,  and  all  elements  in  T  can  be  verified  from 
the  assumptions  in  T,  then  we  can  conclude  A  true  in  T.  As  we  will  see  in  the  next 
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section,  this  transition  from  one  context  \b  to  another  context  T,  can  be  achieved  via 
a  substitutions  from  T  to  T. 


2.2  Syntax 

We  conservatively  extend  LF  [28]  with  modal  variables.  Existential  variables  u  (or  v) 
are  treated  as  modal  variables  and  x  denotes  ordinary  variables.  Existential  variables 
are  declared  in  a  modal  context  A,  while  bound  variables  x:A  are  declared  in  a  context 
T  or  Note  that  the  modal  variables  u  declared  in  A  carry  their  own  context  of 
bound  variables  T  and  type  A.  The  substitution  a  is  part  of  the  syntax  of  existential 
variables,  c  and  a  are  constants,  which  are  declared  in  a  signature. 


Substitutions 

a,r  : 

:=  •  |  cr,  M/x 

Objects 

M,  N  : 

:=  c  |  x  |  u[a]  X x:A.  M  Mi  M2 

Families 

A,B,C  : 

:=  a  |  AM  A2 

Kinds 

K  : 

:=  type  Ihr:A  K 

Signatures 

E  : 

:=  •  |  E,  a:K  E,  c:A 

Contexts 

T,T  : 

:=  •  |  T, x:A 

Modal  Contexts 

A  : 

_L 

s 

4 

We  use  K  for  kinds,  and  A,  B:  C  for  type  families,  M,  N  for  object.  Signatures, 
contexts  and  modal  contexts  may  declare  each  constant  and  variable  at  most  once. 
For  example,  when  we  write  T,x:A  we  assume  that  x  is  not  already  declared  in  T.  If 
necessary,  we  tacitly  rename  x  before  adding  it  to  the  context  T.  Similarly,  when  we 
write  A,'iu:('b  b  A),  we  assume  that  u  is  not  already  declared  in  A.  Terms  that  differ 
only  in  the  names  of  their  bound  and  modal  variables  are  considered  identical. 

2.3  Substitutions 

We  write  the  substitution  operation  as  a  defined  operations  in  prefix  notation  [cr]P, 
for  an  object,  family,  or  kind  P.  These  operations  are  capture-avoiding  as  usual. 
Moreover,  we  always  assume  that  all  free  variables  in  P  are  declared  in  a.  Substitutions 
that  are  part  of  the  syntax  are  written  in  postfix  notation,  u[a\.  Note  that  such 
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explicit  substitutions  occur  only  for  variables  u  labeling  relative  validity  assumptions. 
Substitutions  are  defined  in  a  standard  manner. 


Substitutions 

MK-) 

=  (•) 

W](r,N/y) 

=  (MM  ,W\N/y) 

provided  y  not  declared  or  free  in  a 

Objects 

MM 

=  c 

Mu  M/x,  a2\x 

=  M 

MOM) 

=  u[[a]r] 

MK^iW) 

=  (MW)  ([<t]N2) 

[a]  (Ay:  A  N ) 

=  Ay:  [a]  A.  [a,  y/y\N 

provided  y  not  declared  or  free  in  a 

Families 

[a]a 

=  a 

¥\(AM) 

=  (MM)(M]M) 

[cr](nj/:^4i.  A2) 

=  ny:[u]Ai.  [a,y/y]A2 

provided  y  not  declared  or  free  in  a 

Kinds 

[cr]  type 

=  type 

[a}(Uy:A.  K) 

=  n y\a\A.  [a,  y/y\K 

provided  y  not  declared  or  free  in  a 

The  side  conditions  can  always  be  verified  by  (tacitly)  renaming  bound  variables. 
We  do  not  need  an  operation  of  applying  a  substitution  a  to  a  context.  The  last 
principle  makes  it  clear  that  [u]t  corresponds  to  composition  of  substitutions,  which  is 
sometimes  written  as  r  o  a. 

Lemma  1  (Composition  of  substitution) 


1.  H([t]t')  =  [[ct]t]t' 

2.  [ct]([t]M)  =  [[cr]r]M 

3.  [cr]([r}A)  =  [[<t]t]A 
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4-  W]([t]K)  =  [[<t]t]K 

Proof:  By  simultaneous  induction  on  the  definition  of  substitutions.  □ 

Note  substitutions  o  are  defined  only  on  ordinary  variables  x  and  not  modal  vari¬ 
ables  u.  We  write  idr  for  the  identity  substitution  {x\/x\ , . . .  ,xn/xn)  for  a  context 
T  =  (•,  X\\Ai, . . . ,  xn:An).  We  will  use  tt  for  a  substitution  which  may  permute  the 
variables,  i.e  tt  =  (x$(i)/x\ ,  ■  ■  ■  ,x$(n)/xn)  where  <3>  is  a  total  permutation  and  tt  is 
defined  on  the  elements  from  a  context  T  =  (-,x\\Ai, . . .  ,xn:An).  We  only  consider 
well-typed  substitutions,  so  tt  must  respect  possible  dependencies  in  its  domain.  We 
also  streamline  the  calculus  slightly  by  always  substituting  simultaneously  for  all  ordi¬ 
nary  variables.  This  is  not  essential,  but  saves  some  tedium  in  relating  simultaneous 
and  iterated  substitution.  Moreover,  it  is  also  closer  to  the  actual  implementation 
where  we  use  de  Bruijn  indices  and  postpone  explicit  substitutions. 

A  new  and  interesting  operation  arises  from  the  substitution  principles  for  modal 
variables.  Modal  substitutions  are  defined  as  follows. 

Modal  Substitutions  9  ::=  •  |  9,M/u 

We  write  9  for  a  simultanous  substitution  [Mi /ip , . . . ,  Mn/un]  where  u±, ...  ,un  are 
distinct  modal  variables.  The  new  operation  of  substitution  is  compositional,  but  two 
interesting  situations  arise:  when  a  variable  u  is  encountered,  and  when  we  substitute 
into  a  A-abstraction  (or  a  dependent  type  II  respectively). 


Substitutions 


[9}{<r,N/y)  =  ([0]c,  [9}N/y) 


Objects 


[0i,M/u,02](u[<t]) 
MWAy 
[0](A»:A  N) 


X 

(MAI)  ([#]jv2) 

I#]JV 
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Families 


Kind 


Contexts 


[0]a  =  a 
[0]x  =  x 

W\(am)  =  ([epHMM) 

MtiUA.  a2)  =  rncM/C  {e]A2 


Pltype 

MflUA  K) 


type 

n»:[0]A  ie\K 


P](r,»A)  =  ([0]r,9:[9p) 

We  remark  that  the  rule  for  substitution  into  a  A-abstraction  and  similarly  the  rule 
for  fl-abstraction  does  not  require  a  side  condition.  This  is  because  the  object  M  is 
defined  in  a  different  context,  which  is  accounted  for  by  the  explicit  substitution  stored 
at  occurrences  of  u.  This  ultimately  justifies  implementing  substitution  for  existential 
variables  by  mutation. 

Finally,  consider  the  case  of  substituting  into  a  closure,  which  is  the  critical  case  of 
this  definition. 

[9i,M/u,92}(u[a])  =  [l6l,M/u,92]a]M 

This  is  clearly  well-founded,  because  a  is  a  subexpression  (so  [M/ufcr  will  terminate) 
and  application  of  an  ordinary  substitution  has  been  defined  previously  without  ref¬ 
erence  to  the  new  form  of  substitution.  We  will  continue  the  discussion  of  modal 
variables  in  chapter  3  and  focus  here  on  type-checking  and  definitional  equality  of  the 
conservative  extension  of  LF. 


2.4  Judgments 

We  now  give  the  principal  judgments  for  typing  and  definitional  equality.  We  presup¬ 
pose  a  valid  signature  E,  which  will  be  omitted  for  sake  of  brevity.  If  J  is  a  typing 
or  equality  judgment,  then  we  write  [a\J  for  the  obvious  substitution  of  J  by  a.  For 
example,  if  J  is  M  :  A,  then  [a\J  stands  for  the  judgment  [a]M  :  [cr}A. 
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b 

A  mctx 

A  is  a  valid  modal  context 

A 

b 

d'  ctx 

d<  is  a  valid  context 

A;  T 

b 

a  :  di 

Substitution  a  matches  context  di 

A;  T 

b 

M  :  A 

Object  M  has  type  A 

A;  T 

b 

A  :  K 

Family  A  has  kind  K 

A;  T 

b 

K  :  kind 

K  is  a  valid  kind 

A;  T 

b 

M  =  N  :  A 

M  is  definitional  equal  to  N 

A;  T 

b 

A  =  B  :  K 

A  is  definitional  equal  to  B 

A;  T 

b 

K  =  L  :  kind 

K  is  definitional  equal  to  L 

We  start  by  defining  valid  modal  context  and  valid  context. 


Modal  Context 


b  (•)  mctx 


b  A  mctx  A  b  if  ctx  A;  'I'  b  A  :  type 
b  (A,  w::('l/i-bL))  mctx 


Context 


A  b  (•)  ctx 


A  b  $  ctx  A;$bd:  type 
A  b  (d/,  x:A)  ctx 


Note  that  there  may  be  dependencies  among  the  modal  variables  defined  in  A. 


2.5  Typing  rules 

In  the  following,  we  will  concentrate  on  the  typing  rules  and  definitional  equality  rules 
for  substitutions,  objects,  types  families,  and  kinds.  We  will  follow  the  formulation  of 
the  typing  rules  given  in  [29].  We  presuppose  that  all  modal  contexts  A  and  bound 
variable  context  Y  in  judgments  are  valid. 
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Substitutions 

A;rhcr:$  A-,ThM:[a]A 
A;T  b  (•)  :  (•)  A;T  b  (a,  M/x)  :  (T,a;:Al) 


Object 


A,w::(Tm4),  A';r  b  a  :  T 

var  - - — — 77 — . mvar 


A;  T,  x:A,  T'  b  x  :  A  A;  w::(\&kA),  A';  V  b  u[a]  :  [a \A 


A;  r,  x:Ai  b  M  :A2  A;  T  b  Ax  :  type  A;  Y  b  Ml  :  Ux:A2.  A;  T  b  M2  :  A2 


A;T  b  AxiAx.  M  :  rixiAx.  A2 


A;  T  b  Mi  M2  :  [idr,  M2/x\Ax 


A;  r  b  M  :  B  A;  T  b  A  =  B  :  type 
A;  r  b  M  :  A 


conv 


Families 

<r/l  in  E  A;  T  b  A  :  Tlx:B.K  A;  T  b  M  :  B 
A;  T  b  a  :  K  A;  Y  b  A  M  :  [idr,  M/x]K 

A;  T  b  Ai  :  type  A;  T,  x:Ai  b  A2  :  type  A;  T  b  A  :  K  A;  T  b  K  =  L  :  kind 
A;  T  b  IIt:Ai.742  :  type  A;  T  b  A  :  L 


Kind 

A;  T  b  A  :  type  A;  T,  xiA  b  K  :  kind 
A;  T  b  type  :  kind  A;  T  b  Ylx:A.K  :  kind 

Note  that  the  rule  for  modal  variables  is  the  rule  (*)  presented  earlier,  annotated 
with  proof  terms  and  slightly  generalized,  because  of  the  dependent  type  theory  we 
are  working  in.  This  rule  also  justifies  our  implementation  choice  of  using  existential 
variables  only  in  the  form  u[cr\. 


2.6  Definitional  equality 

Next,  we  give  some  rules  for  definitional  equality  for  objects,  families  and  kinds  in  A. 
Some  of  the  typing  premises  marked  as  {. . .}  are  redundant,  but  we  cannot  prove  this 
until  validity  has  been  established.  We  do  not  include  reflexivity,  since  it  is  admissi- 
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blc.  The  interesting  case  is  the  one  for  modal  variables.  Two  modal  (or  existential) 
variables  are  considered  definitional  equal,  if  they  actually  are  the  same  variable  and 
the  associated  substitutions  are  definitionally  equal.  This  means  that  if  we  implement 
existential  variables  via  references,  then  two  uninstantiated  existential  variables  are 
only  definitional  equal  if  they  point  to  the  same  reference  under  the  same  substitution. 


Substitutions 

A;T  b  M  =  N  :  [a]A  A;  T  b  a  =  o'  :  T 
A;  T  b  •  =  •  :  •  A;  T  b  (a,  M/x)  =  (a',N/x)  :  (T,  x:A) 

Simultaneous  Congruence 

A;  T  b  A  =  A'  :  type  A;  T  b  A"  =  A'  :  type  A;  T,  x:A  b  M  =  N  :  B 
A;  T  b  Xx-.A'.M  =  \x:A".N  :  Ylx:A.B 

A;  T  b  Mx  =  Nx  ■  n x:A2.  Ai  A;  T  b  M2  =  N2  :  A2 
AT  b  Mi  M2  =  Ni  N2  :  [idr,  M2jx\Ax 

A,u::(ThA),  A';T  b  a  =  a'  :  T 
A,  u::(ThA),  A';  T  b  u[a\  =  u[a']  :  [u]A 

c:A  in  signature  E 

A;  T,  x:A ,  T'  b  x  e  i :  d  A;  T  b  c  e  c  :  d 


Parallel  conversion 

{A;  T  bd  :  type}  A;T,x:dbM2EiV2:5  A;  T  b  Mx  =  Nx  :  d 
A;  T  b  (A x:A.M2)M1  =  [id^Ni/x]^  :  [idr,  M1/x]B 


Extensionality 

{A;T  b  N  :  UxiA^A^ 

(A;  TbM:  IItnAi.^}  A;  T  b  A\  :  type  A;  T,  x\A\  b  M  x  =  A  x  :  A2 

A;T  b  M  e  A  : 
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Equivalence 

A;  T  h  N  =  M  :  A  A;  T  h  M  =  N  :  A  A;  T  h  N  =  O  :  A 
A;  T  h  M  =  N  :  A  A:  Y  F  M  =  0  :  A 

Type  conversion 

A;  r  h  M  =  N  :  A  A;  T  h  A  =  B  :  type 
A;  T  h  M  =  N  :  B 

Family  congruence 

a:K  in  E  A;  T  h  A  =  B  :  Ux:C.K  A;  T  h  M  =  N  :  C 

A;  r  F  a  =  a  :  K  A;  T  h  A  M  =  B  N  :  [idr,  M/x\K 

A;  T  h  Ai  =  Bi  :  type  {A;  T  h  A\  :  type}  A;  T,  x:Ai  h  A2  =  B2  :  type 
A;  T  h  nrr:A1.A2  =  Hx:Bi.B2  :  type 

Family  equivalence 

A;  T  h  A  =  B  :  K  A;  T  h  A  =  B  :  K  A;  T  h  B  =  C  :  K 
A;  T  h  B  =  A:K  A;  T  h  A  =  C  :  K 

Kind  conversion 

A;  T  h  A  =  B  :  K  A;  T  h  K  =  L  :  kind 
A;T  h  A  =  B  :  L 

Kind  congruence 

A;  T  h  type  =  type  :  kind 

A;  T  h  A  =  B  :  type  (A;  T  h  A  :  type}  A;  T,  x:A  h  K  =  L  :  kind 
A;T  h  =  IIxi-B.L  :  kind 

Kind  equivalence 

A;T  h  K  =  L  :  kind  A;  Y  h  Jl  =  V  :  kind  A;  T  h  L' =  L  :  kind 
A;  T  h  L  =  A’  :  kind  A;  T  h  I<  =  L  :  kind 
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2.7  Elementary  properties 

We  will  start  by  showing  some  elementary  properties. 

Lemma  2  (Weakening) 

1.  If  A;  T,  P  b  J  then  A;  T,  x:A,  P  b  J 

2.  If  A,  A';r  b  J  then  A,  u::(tfhA),  A';  T  b  J 

Proof:  Proof  by  induction  over  the  structure  of  the  given  derivation.  □ 

Next,  we  show  that  reflexivity  is  admissible. 

Lemma  3  (Reflexivity) 

1.  //  A;  T  h  <T  :  $  then  A;  Y  b  cr  =  cr  :  ^ 

2.  If  A;  T  h  M  :  A  t/ien  A;  T  h  M  =  M  :  A 

3.  If  A;  T  h  A  :  Jl  then  A;T\~  A  =  A:  K 

4.  If  A;  T  h  A'  :  kind  t/mn  A;  T  h  A'  =  A'  :  kind 

Proof:  Proof  by  induction  over  the  structure  of  the  given  derivation.  □ 

First,  we  show  some  simple  properties  about  substitutions,  which  will  simplify  some 
of  the  following  proofs.  We  always  assume  that  the  contexts  for  bound  variables  T  and 
are  valid.  Similarly,  the  modal  context  A  is  valid. 

Lemma  4  Let  A;  T  h  <j\  :  fl/ 1  and  A;  T  b  (ui,  <r2)  :  (d'l,  'P2)- 

1.  If  A;  'I'!  h  r  :  'P  then  [<7i,<72](t)  =  [<7i](t). 

2.  //AAihM:  A  then  [cri,cr2]M  =  [<ri](M)  and  [cri,cr2]A  =  [<Ti](A). 

3.  //A;  h  A  :  K  then  [<Ti,<72]A  =  [cr1](A)  and  [ai,(T2]A'  =  [ai](A'). 

4 ■  If  A;  'll!  b  K  :  kind  then  [ai,a2]K  =  [cr1](A'). 


29 


CHAPTER  2.  DEPENDENTLY  TYPED  LAMBDA  CALCULUS  BASED  ON  MODAL  TYPE  THEORY 


5.  If  A;  T  b  idp  :  T  and  A;Th  a  :  $  then  [idr]  (cr)  =  cr 

6.  If  A;  T  b  cr  :  4/  and  A;  4/  b  id : T  then  [cr]  (id^,)  =  cr 

Proof:  The  statement  (1)  to  (4)  are  proven  by  induction  on  the  first  derivation.  The 
statement  (5)  by  induction  on  the  derivation  A;  T  b  cr  :  T  and  the  last  one  by  induction 
on  the  size  of  id^  using  statement  (1).  □ 

Lemma  5  (Inversion  on  substitutions) 

If  A;  T  b  (cr ! ,  N/x,  cr2)  :  (Tx,  x  :  A,  T2)  then  A;  T  b  A  :  [<Ji]A  and  A;  T  b  cy  :  Tx 
Proof:  Structural  induction  on  rr2.  □ 

Lemma  6  (General  substitution  properties) 

1.  If  A;  T  b  cr  :  T  and  A;  $  h  r  :  T  then  A;  T  b  [cr]r  :  Tb 

2.  If  A;  T  b  cr  :  T  mid  A;  T  b  T\  =  r2  :  4/'  then  A;  T  b  [ct]ti  =  [cr]r2  :  Tb 

3.  //  A;  T  b  cr  :  T  and  A;  T  b  A  :  C  fden  A;  T  b  [cr]  A  :  [<r]C. 

//  A;  T  b  a  :  T  and  A;  T  b  A  =  M  :  A  then  A;  T  b  [cr]  A  =  [ a]M  :  [cr]  A. 

5.  //  A;  T  b  cr  :  T  and  A;  T  b  A:  K  then  A;  T  b  [a]A  :  [a]K. 

6.  //  A;  T  b  cr  :  T  and  A;  T  b  A  =  B  :  K  then  A;  T  b  [a]A  =  [a]B  :  [a]K. 

7.  //  A;  T  b  cr  :  T  and  A;  T  b  K  :  kind  then  A;  T  b  [<j\K  :  kind. 

8.  //  A;  T  b  cr  :  T  and  A;  T  b  K  =  L  :  kind  fden  A;T  b  [cr] A  =  [cr]L  :  kind. 

Proof:  By  simultaneous  induction  over  the  structure  of  the  second  derivation.  We 
give  some  cases  for  (3). 

Case  V  = _ 

A;  (Tx,a;:A,T2)  b  x  :  A 


A;  Tx  b  A  :  type 

A;  T  b  (ay,  N/x,  cr2)  :  Tx,  x:A,  T2 
A;  T  b  A  :  [a^A  and  A;  T  b  oy  :  Tx 
A;  T  b  A  :  [al,N/x,a2]A 


by  validity  of  ctx  T 
by  assumption 
by  lemma  5 
by  lemma  4 


30 


2.7.  ELEMENTARY  PROPERTIES 

Case  V  =  A;  h  M  :  A2  A;  'I'  h  A  :  type 

A;  'P  b  A x:Ai.M  :  nrc:A1.A2 

A;T  b  a  : ^ 

by  assumption 

A;  T  b  [cr]  Ai  :  type 

by  i.h. 

a  b  r  ctx 

by  asumption 

A  b  T,  x:\a\A\  ctx 

by  rule 

A;  T,  x:[cr]Ai  b  a  :  'b 

by  weakening 

A;  T,  x :  [<y]Ai  b  x  :  [cr] A\ 

by  rule 

A;  T,  x:\a\Ai  b  (a,x/x)  : 

by  rule 

A;  T,  x\[a\Ai  b  [cr,  x/x]M  :  [a,x/x]A2 

by  i.h. 

A;  T  b  Ax:[cr]Ai.[cr,  x/x]M  :  IIx:[cr]Ai.[cr,  x/x]A2 

by  rule 

A;  T  b  [cr](\x:Ai.M)  :  [cr]  (IIx:Al.A2) 

by  subst. 

definition 

Case  V  =  A;  ^  h  Ml  :  Ux:A^A^  A;$b  M2  :  A2 

A;  V  b  (Afi  M2)  :  [id®,  M2/x]A! 

A;Tb  [a]Mi  :  [<t]Hc:A2.Ai 

by  i.h. 

A;  T  b  [cr] Afx  :  IIx:[cr]A2.[cr,  x/xJAx 

by  subst. 

definition 

A;  T  b  [ct]M2  :  [ct]A2 

by  i.h. 

A;  T  b  ([cr]Mi)  ([cr]M2)  :  [idr  Jcr]M2/x]  ([cr,  x/x]Ai) 

by  rule 

[idr,  [cr]M2/x](cr,  x/x)  =  ([idp,  [cr]M2/x]cr,  [cr]M2/x) 

by  subst. 

definition 

=  (H  [ct]M2/x) 

by  lemma  4 

=  (M(id®),  [cr]M2/ x) 

by  lemma  4 

=  [cr]  (id®,  M2/x) 

by  subst. 

definition 

A;T  b  [a](A/!  M2)  :  [cr]([id®,  M2/x]A0 

by  subst. 

definition 

Case  T)  =  A.^t.A';  ®  hr  :  ^ 

A,  ■u::'k1hA1,  A';  $  b  m[t]  :  [rjAb 

A,M::^1hA1,A';rb([cr]T):^1 

by  i.h. 

A,  ■u::4'1hA1,  A';  T  b  «[[tr]r]  :  [[a]r]A! 

by  rule 
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A,  m-'I'ikAi,  A';  T  b  [cr](tt[r])  :  [cr]([r]Ai)  by 


Case  V  = 


A;  $  h  M  :  B  A;  \l>  b  B  =  A  :  type 
A;^  b  M  :  A 


lemma  1  and  subst.  definition 


A;  r  b 

[a]M  : 

[a]B 

by  i.h. 

A;  T  b 

W)B  = 

[o]A  :  type 

by  i.h. 

A;  T  b 

[a}M  : 

[a]A 

by  rule 

□ 

Lemma  7  (Renaming  substitution)  A;r ,y:A  b  (\dr,y/x)  :  (r,a;:A). 


Proof: 

A;  T  b  idr  :  T  by  definition 

A;  T ,y:A  b  idr  :  T  by  weakening 

A;  T,  y:A  b  y  :  A  by  rule 

A;  Y,y:A  b  y  :  [idr]A  by  definition 

A;  T,y:A  b  (\dr,y/x)  :  (r,x:A)  by  rule 


□ 


Lemma  8  (Context  Conversion) 

Assume  Y,x\A  is  a  valid  context  and  V  b  B  :  type. 

If  A;  T,  x:A  b  J  and  A;  Y  b  A  =  B  :  type  then  A;  Yx:B  b  J . 


Proof:  direct  using  weakening  and  substitution  property  (lemma  6). 

A;  T,  x:B  b  x  :  B  by  rule 

A;  T  b  B  =  A  :  type  by  symmetry 

A;r,r/:A  b  (id r,y/x)  :  (r,a;:A)  renaming  substitution 

A;  Y,y.A  b  [id r,y/x]J  renaming  of  assumption 

A]Y,x:B,y:AY  [idr ,y/x\J  weakening 

A]Y,x:B  b  (\dr,x/x)  :  (Y,x:B)  by  definition 

A;  Y,x:B  b  B  =  A  :  type  by  weakening 
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A;  T ,x:B  b  x  :  A 

A;  T,  x:B  b  x  :  [idp,  a;/a;]A 

A;T,a::i?  b  (idr,  x/x,  x/y)  :  ( T,x:B,y:A ) 

A;T,a::fi  b  [idr,  x/x,  x/7/]([idr,  y/x]  J) 

A-,T,x:B  b  [idp  ,  x/x\J 

A;  T,  x:B  b  J 


by  rule  (conv) 
by  subst.  definition 
by  rule 

by  substitution  property 
by  subst.  definition  and  lemma  4 
by  snbst.  definition 


□ 

Next,  we  prove  a  general  functionality  lemma  which  is  suggested  by  the  modal 
interpretation. 

Lemma  9  (Functionality  of  typing  under  substitution)  Assume  A;T  b  <r  : 

A;  T  b  b  :  $  and  A;  T  b  a  =  a'  :  4'. 

1.  If  A;  'll  b  r  :  4b  then  A;  T  b  ([u]t)  =  ([cbjr)  :  4b 

2.  If  A;  ^  b  M  :  A  then  A;  T  b  [a]M  =  [a']M  :  [a]A. 

3.  If  A;  ^  b  A  :  AT  t/ien  A;  T  b  [a]  A  =  [a']  A  :  [a]K. 

4-  If  A;  4'  b  A'  :  kind  then  A;  T  b  [u]K  =  [a']  A”  :  kind. 

Proof:  Simultaneous  induction  on  the  given  derivation.  First,  the  proof  for  (1). 

Case  V  = _ 

A;rb-:- 

A;  r  b  •  =  •  :  • 

A;  r  b  [cr] (•)  =  [cr']  (-) 

V 1  v2 

Case  v  =  ^^N-.\r\A  A;j.hr:P 

A;f  h  (r,  N/x)  :  ('S'^x-.A) 

A;  T  b  ([<t]t)  =  ( [a']T )  :  4b  by  i.h.  on  V2 


by  rule 
by  subst.  definition 
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A;  T  h  [a]N  =  [cr']N  :  [ct]([t]A) 

A;  T  b  [a]N  =  [a']N  :  [[(t]t]A 

A;  T  b  ([<t]t,  [ a\N/x )  =  ([<t']t,  [cr']N/x)  :  (^axA) 

A;  T  b  \<j\(t,N/x)  =  [<j'](t,N/x)  :  (4b,  mA) 


by  i.h.  on  T>\ 
by  lemma  1 
by  rule 
by  subst.  definition 


Next,  the  proof  for  (2).  We  only  show  the  case  for  modal  variable  u. 

Oase  v  =  Ai.rfWi, 

A1;  u::'$,'\-A,  A2;  bu[r]  :  [t\A 

A2;T  b  ([cr]r)  =  ([< t']t )  : 

Ai,  ur.'Sf'hA,  A2;  T  b  «[[ct]t]  =  ziffcr'Jr]  :  [[crjrjA 
Ai ,u::'f?'\-A,  A2;T  b  [«r](«[r])  =  [a'](u[r])  :  [u]([r]A) 


□ 


by  i.h. 
by  rule 
by  lemma  1 
and  subst.  definition 


Lemma  10  (Inversion  on  products) 

1.  If  A;  T  b  Ihr:bLi.bl2  :  K  then  A;  Y  b  A\  :  type  and  A;  T,  x\A\  b  A2  :  type 

2.  If  A;  T  b  Yix\A.K  :  kind  then  A;  T  b  A  :  type  and  A;  T,  x:A\  b  K  :  kind 

Proof:  Part  (1)  follows  by  induction  on  the  derivation.  Part  (2)  is  immediate  by 
inversion.  □ 

Lemma  11  (Validity) 

1.  If  A;  r  b  M  :  A  then  A;  T  b  A  :  type. 

2.  If  A;  T  b  A  :  K  then  A;  T  b  K  :  kind. 

3.  If  A;  T  b  a  =  a’  :  then  A;  V  b  a  :  'P  and  A;T  b  b  :  $ 

4.  If  A;  r  b  M  =  N  :  A,  t/ien  A;  T  b  M  :  A,  A;  T  b  V  :  A  and  A;  T  b  A  :  type. 

5.  If  A;  T  b  A  =  B  :  K,  thenA ;  r  b  A  :  K,  A;  T  b  5  :  K  and  A;  T  b  K  :  kind. 
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6.  If  A;T  b  K  =  L:  kind,  then  A;  T  b  K  :  kind.  A;  T  b  L  :  kind. 

Proof:  Simultaneous  induction  on  derivations.  The  functionality  lemma  is  needed  for 
application  and  modal  variables.  In  addition,  the  cases  for  modal  variables  use  the 
validity  of  the  modal  context  A.  The  typing  premises  on  the  rule  of  extensionality 
ensure  that  strengthening  is  not  required.  First,  we  show  the  case  for  modal  variables 
in  the  proof  for  (1). 

Case  V  =  A.rfTAdHc:* 

A,  u::ThA,  A';  T  b  u[a\  :  [<j\A 


A;  T  b  A  :  type 
A,  u::ThA,  A';  T  b  A  :  type 
A,u::ThA,  A';  T  b  [<j\A  :  type 


by  validity  of  mctx 
by  weakening 
by  lemma  6 


Consider  the  proof  for  (4). 

T>  i 

^  A,  u::ty\-A,  A7;  T  b  a  =  a'  :  T 

ise  V  = _ ! _ ’  ’ _ 

A,  A';  T  b  u[a]  =  u[a']  :  [cr]A 

A,w::ThA,  A';  T  b  a  :  T 
A,  u::ThA,  A';  T  b  u[a\  :  [<j\A 
A,u::ThA,  A';  T  b  a’  :  T 
A,  u::Ti-A,  A';  T  b  u[a']  :  [a ']A 
A;  T  b  A  :  type 
A,  u::ThA,  A';  T  b  A  :  type 
A,  M::ThA,  A';  T  b  A  =  A  :  type 
A,u::^hA,  A';  T  b  a'  =  tr  :  T 
A,u::ThA,  A';  T  b  [cr']A  =  [<t]A  :  type 
A,  u::ThA,  A';  T  b  ufa']  :  [u\A 


by  i.h. 
by  rule 
by  i.h. 
by  rule 
by  validity  of  mctx 
by  weakening 
by  reflexivity  (lemma  3) 
by  symmetry 
by  functionality  (lemma  9) 
by  type  conversion 


T>  l  V  2  V  3 

Case  V  =  r  ^  Al  :  type  r’ ^  M‘2  =  ^2  :  A2  A;  Tb  Mi  =  Ah  :  A, 
A;  T  b  (A x-.AM)  Mi  =  [idr,  Ah/x]  A2  :  [idr,  M^x]^ 
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A;  T,  x\A\  b  A2  :  type 
A;  r ,  x:A\  b  AI2  '■  A2 
A;  T  b  A x:Ai.M2  :  Tix:Ai.A2 
by  rule 

A;  r  b  Ml  :  Ax 

A;  r  b  (A x:AvM2)  M1  :  [id T,M1/x\A2 

A;  r ,  x:A\  b  N2  :  A2 

A;  T  b  Afi  :  Ax 

A;  r  b  idr  :  r 

A;  r  b  Ai  :  [idp]Ai 

A;  T  b  (idr,  N1/x)  :  (r,  xiA,) 

A;  T  b  [idp,  :  [\dp,  Ni/x]A2 

A;  T,  x\Ai  b  A2  =  A2  :  type 

A;  r  b  idp  ==  idp  :  T 

A;  r  b  Mx  =  N1:A1 

A;  r  b  Mx  =  A,  :  [idr]A 

A;  r  b  (idp,  Mjx)  =  (idp,  Njx)  :)r, 

A;  T  b  [idp,  M1/x\A2  =  [idp,  Ni/x]A2  :  type 
A;  T  b  [idp,  Ni/x]A2  =  [idp,  M1/x]A2  :  type 
A;T  b  [idp,  Njx]^  :  [idp ,Mi/x\A2 
A,r  b  idp :  r 
A;  r  b  Ml  :  Ax 
A;  r  b  Ml  :  [idpjAl 
A;  r  b  (idp,  Mi/x)  :  r,x:A 
A;  T  b  [idp,  M1/x\A2  :  type 


by  i.h. 
by  i.h. 


by  i.h. 
by  rule 
by  i.h. 
by  i.h. 
by  definition 
by  definition 
by  rule 

by  substitution  (lemma  6) 
by  reflexivity  (lemma  3) 
by  defintion 
by  assumption  T>3 
by  definition 
by  defintion 
by  functionality  lemma  9 
by  symmetry 
by  type  conversion 
by  definition 
by  i.h. 
by  definition 
by  rule 

by  substitution  lemma  6 

□ 


Lemma  12  (Typing  Inversion) 

1.  If  A;  T  b  x  :  A  then  x\B  in  Y  and  A;  Y  b  A  =  B  :  type  for  some  B. 

2.  If  A;  T  b  c  :  A  then  c:B  in  E  and  A;  Y  b  A  =  B  :  type  for  some  B. 
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3.  If  A;  T  b  M\  M2  :  A  then  A;T  h  M\  :  Yix-.A2.A1,  A;T  h  M2  :  A2  and 
A;  T  b  [idr,  M2/x\Ai  =  A  :  type  for  some  A\  and  A2. 

4-  If  A;  T  b  Xx:A.M  :  B  then  A;  T  b  R  =  Yix\A.A'  :  type  and  A;r,a::A  h  M  :  i' 
and  A;  T  b  A  :  type  for  some  A  and  A' . 

5.  If  A;  T  b  u[a]  :  B  then  (Ai,  ur'YhA,  A2)  =  A  and  Ai,  uv.’YeA,  A2;  Y  b  cr  :  ^  and 
Al5  ur.'YrA,  A2;  Y  b  B  =  [<r]A  :  type  for  some  vb,  A,  Ai  and  A2. 

5.  If  A;  T  b  IIai:A1.A2  :  A'  t/jen  A;  T  b  A'  =  type  :  kind.  A;  Y  b  Ai  :  type  and 
A;  T,  a::Ai  b  A2  :  type. 

7.  If  A;  T  b  a  :  Jl  i/ien  a:L  in  E  and  A;  T  b  Jl  =  L  :  kind  /or  some  L. 

5.  If  A;  T  b  A  M  :  iAen  A;  Y  b  A  :  A;  T  b  M  :  A1  and 

A;  T  b  K  =  [idr,  M/x\K2  :  kind  for  some  Ai  and  K2. 

Proof:  By  straightforward  induction  on  typing  derivations.  Validity  is  needed  in  most 
cases  in  order  to  apply  reflexivity.  □ 

Lemma  13  (Redundancy  of  typing  premises)  The  indicated  typing  premises  in 
the  rules  of  parallel  conversion  and  instantiation  of  modal  variables  are  redundant. 

Proof:  By  inspecting  the  rules  and  validity.  □ 

Lemma  14  (Equality  inversion) 

1.  If  A;  T  b  K  =  type  :  kind  or  A;  Y  b  type  =  K  :  kind  then  K  =  type. 

2.  If  A;  T  b  K  =  IIa::i?i.L2  :  kind  or  A;  Y  b  Ylx\Bi.L2  =  K  :  kind  then 

K  =  Yix\Ai.K2  such  that  A;T  b  A\  =  B\  :  kind  and  A;r,a;:Ai  b  Jl2  =  L2  :  kind. 

3.  If  A;  T  b  A  =  Yix.B1.B2  :  type  or  A;  Y  b  Yix.B1.B2  =  A  :  type  then 

A  =  for  some  Ax  and  A2  such  that  A;T  b  Ai  =  Bi  :  type  and 

A;  T,  x\Ai  b  A2  =  B  :  type. 


Proof:  By  induction  on  the  given  equality  derivations. 


□ 
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Lemma  15  (Injectivity  of  Products) 

1.  If  A;  T  b  Yix:Ai.A2  =  Ylx:Bi.B-2  :  type  then  A;T  b  A\  =  B\  :  type  and 
A;  T,  x:Ax  \~  A2  =  B2  '■  type. 

2.  If  A;  T  b  =  Tlx:Bi.L2  :  kind  then  A;T  b  Ai  =  Bi  :  type  and 

A;  T,  x\Ai  b  K2  =  L2  :  kind. 


Proof:  Immediate  by  equality  inversion  (lemma  14). 


□ 


2.8  Type-directed  algorithmic  equivalence 

One  important  question  in  practice  is  whether  it  is  still  possible  to  effectively  decide 
whether  two  terms  are  dehnitionally  equal.  In  [29],  Harper  and  Pfenning  present  a 
type-directed  equivalence  algorithm.  We  will  extend  this  algorithm  to  allow  modal 
variables.  Crucial  in  the  correctness  proof  for  the  algorithmic  equality  in  [29]  is  the 
observation  that  we  can  erase  all  dependencies  among  types  to  obtain  a  simply  typed 
calculus  and  then  show  that  algorithm  for  equality  is  correct  in  this  simply  typed 
calculus.  This  idea  carries  over  to  the  modal  extension  straightforwardly.  Following 
[29],  we  write  a  for  simple  base  types  and  have  a  special  type  constants,  type-. 


Simple  Kinds  k  ::=  type-  |  r  — >  k 

Simple  Types  r  ::=  a  j  T\  — >•  t2 

Simple  contexts  fl,  <F  ::=  •  j  x:t 

Simple  modal  contexts  A  ::=  ■  |  A,  i:($  b  r) 

We  write  A~  for  the  simple  type  that  results  from  erasing  dependencies  in  A,  and 
similarly  K~. 
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(a)  =  a 

(AM)~  =  A~ 

( T\x:A\.A2 )“  =  Af  — >  A~2 

type-  =  type- 

(n x:A.K)~  =  A-  -»•  Jl- 

(■)"  =  • 

(r,  x-.A)~  =  r-,  x:A- 
(•)-  =  • 

(A,  =  A-,  r(rhA-) 

(kind)-  =  kind 

Lemma  16  (Erasure  preservation) 

1.  If  A;  T  b  A  =  B  :  K  then  A~  =  B~ . 

2.  If  A;  T  h  K  =  L  :  kind  then  K~  =  L~ . 

3.  If  A;  r  h  B  :  K  and  A;  \1/  h  cr  :  T  then  B~  =  [a]B~ . 

f.  If  A;  T  b  Jl  :  kind  and  A;  $  h  a  :  T  then  K~  =  [a]K~ . 

5.  If  A;  T  h  idp  :  T  then  idp  =  idp-. 

Proof:  By  induction  over  the  structure  of  the  given  derivation.  □ 

The  following  four  judgments  describe  algorithmic  equality: 

A;  Q  b  M  M'  M  weak  head  reduces  to  M' 

A;0  h  M  <==>■  N  :  t  M  is  equal  to  N 

A;0  h  M  < - >  N  :  t  M  is  structurally  equal  to  N 

A;  12  b  a  < - >  o'  :  a  is  structurally  equal  to  o' 

For  the  weak  head  reduction,  it  is  not  strictly  necessary  to  carry  around  A  and 
explicitcly,  but  it  will  make  the  weak  head  reduction  rules  more  precise.  Next,  we  give 
the  type-directed  equality  rules. 
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Substitution  equality 

A;n\-a< - A;flhMy^iV:r 

A;  f]  b  •  4 - *  •  :  •  A;  O  h-  (cr,  M/x)  4 - *  (cb,  N/x )  :  ($,  x:t) 

Weak  head  reduction 

A;Oh  (A  x-.Ax.M2)  Mi  ^  [idn,  M1/x]M2 

Type-directed  object  equality 

A;fihM*  M1  A;OhM'^A;a  A;  ft  b  N  ^  N'  A;fihI^A;a 
A;f2  b  M  <?=>  N  :  a  A;  Q  b  M  N  :  a 

A;  O  b  M  4 - ■»  N  :  a  A;  Q,  x:t±  b  M  x  •<=>■  N  x  :  t2 

A;  Q  b  M  <^=>-  N  :  a  A;  fi  b  M  <*=*►  iV  :  n 

Structural  object  equality 

o;:t  in  1]  _ c:A  in  S _ 

A;0  b  1  4 - »  x  :  t  A;H  b  c  4 - »  c  :  A- 

b  r)  in  A  A;  f2  b  cr  4 - 4  a'  :  $ 

A;  Q  b  u[a\  4 - >  u[a'\  :  r 

A;  n  b  Mi  4 - >  Ad  ;  T2  — 4  Ti  A;  b  M2  Ad  :  r2 

A;Ob  Mi  M2  4 - ►  Ad  Ad  77i 

Note  that  in  the  rule  *,  we  do  not  apply  a  to  the  type  r.  Since  all  dependencies 
have  been  erased,  r  cannot  depend  on  any  variables  in  f2.  The  algorithm  is  essentially 
deterministic  in  the  sense  that  when  comparing  terms  at  base  type,  we  first  weakly 
head  normalize  both  sides  and  then  compare  the  results  structurally.  Two  modal 
variables  are  only  structurally  equal  if  they  actually  are  the  same  modal  variable.  This 
means  that  if  we  implement  existential  variables  via  references,  then  two  uninstantiated 
existential  variables  are  only  structurally  equal  if  they  point  to  the  same  reference.  This 
algorithm  closely  describes  the  actual  implementation  and  the  treatment  of  existential 
variables  in  it.  We  mirror  these  equality  judgements  on  the  family  and  kind  level. 


A;Ob  Mi  ^  M[ 
A;Ob  Mi  M2  ^  M[  M2 
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Kind-directed  object  equality 

A;  0  b  A  < - ■>  B  :  type-  A;  O,  x:r  b  Ax  -<=>•  B  x  :  k 

A;Ohd  B  :  type-  A;  O  b  A  -<=^  B  :  r  — >  k, 

Structural  family  equality 

a:K  in  E  A;  0  b  A  * - >  B  :  r  — >  n  A;  0  b  M  •<=>■  N  :  r 

A;fiba< - >  a  :  K~  A;  0  b  AM  < - >  BN  :  k 

Algorithmic  kind  equality 

A;  0  b  A  B  :  type-  A;  0  b  K  L  :  kind- 
A;  O  b  type  •<=>■  type  :  ki nd  A;  O  b  II x:A.K  Ux:B.L  :  kind 

The  algorithmic  equality  satisfies  some  straightforward  structural  properties,  such 
as  exchange,  contraction,  strengthening,  and  weakening.  Only  weakening  is  required 
in  the  proof  of  its  correctness. 

Lemma  17  (Weakening) 


1.  If  A;  0,0'  b  a  < - >  o'  :  O"  then  A;  0,x:t,  O'  b  o  < - *  o'  :  0". 

2.  If  A;  O,  O'  b  M  -4=>-  N  :  k  then  A;  O,  x:t,  fl'bM  •<=>■  N  :  k. 

3.  If  A;  O,  O'  b  M  < - >  N  :  k  then  A;  O,  x:t,  O'  b  M  < - ■»  N  :  k. 

f.  If  A;  fi,  fi'  b  d  B  :  hi  then  A;  0,  x:r,  O'  b  A  B  :  hi 

5.  If  A;  O,  O'  b  A  < - »  B  :  hi  then  A;  O,  x:r,  O'  b  A  * - >  B  :  n 

6.  If  A;  0,0'  b  K  -<=4*  L  :  kind  then  A;  0,x:t,  O'  b  K  L  :  kind 

To  show  the  above  extension  to  the  equality  algorithm  is  correct,  we  can  follow  the 
development  in  [29].  We  first  consider  determinacy  of  algorithmic  equality. 

Theorem  18  (Determinacy  of  algorithmic  equality) 
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1.  M1  and  A;  ft  h  M  ^  M2  then  M1  =  M2. 

2.  //  A;  0  I—  M  * — >  N  :  r  then  there  is  no  M'  such  that  A;0  h  M  M' . 

3.  //'  A;  O  b  M  < — >  N  :  r  then  there  is  no  N'  such  that  A;  fl  h  N  N' . 

f.  If  A;  fl  b  M  * — >  N  :  r  and  A;  Vt  h  M  * — ■>  N  :  t'  then  r  =  r1 . 

5.  If  A]  Q  \~  A  < — >  B  :  k  and  A;Qhi  <■ — >  B  :  k'  then  k  —  k'. 

Proof:  The  proofs  follow  [29].  Proof  of  (1)  is  a  straightforward  induction  on  the 
derivation.  For  (2),  we  assume 

5  w 

A;Oh  M  * - >  N  :  t  and  A;  fl  h  M  ^  M' 

for  some  M' .  We  show  by  induction  over  S  and  tV  that  these  assumptions  are  con¬ 
tradictory.  Whenever,  we  have  constructed  a  judgment  such  that  there  is  no  rule  that 
could  conclude  this  judgment  we  say  we  obtain  a  contradiction  by  inversion. 

Case  5  =  X-T  in  9. 

A;  02  b  x  < - »  x  :  t 


A;  02  b  x  —>  M'  by  assumption 

Contradiction  by  inversion 


Case  S  = _ 

A,  ■u::<Pi-t,  A';  02  h  u[a]  < - >  u[a\  :  r 

A,  u::<f>hT,  A';  It  h  u[a\  M'  by  assumption 

Contradiction  by  inversion 


Si 

Case  $  =  A;  ^  P  Mi  <■ — >  Nx  :  r2  — >  n  A;  02  h  M2  4=^  N2  :  r2 

A;  Oh  M,  M2  < - *  N{  N2  :  r, 


A;Oh  Mi  M2  ^  M' 


by  assumption 
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Sub-case  W  = - 

A;U  b  (A x:A1.M\)M2  ^  [idn,  M2/x]M\ 

Ml  =  Xx:Ai.M[ 

A;  It  h  (A x:Ai.M[)  < - ■>  N2  :  r2  — »  Ti 

contradiction 

Wi 

A;  n  h  Ml  ^  m; 

Sub-case  W  = - 

A;OhMj  M2  ^  M[  M2 

contradiction  by  i.h.  on  d>i  and  Wi 

□ 

Using  detcrminacy,  we  can  then  prove  symmetry  and  transitivity  of  algorithmic 
equality. 

Theorem  19  (Symmetry  of  algorithmic  equality) 

1.  If  A]  Q  \~  cr  < - -»  o'  :  <f>  then  A;  12  b  o'  * - *  o  :  <f>. 

2.  If  A;0  h  M  < - *  N  :  r  then  A;  ft  b  N  < - ■>  M  :  t. 

3.  //A;OhM^A:r  then  A;hh  A  4=4  M  :  t. 

f.  If  A;  ft  b  A  < *  B  :  k  then  A;OhB  4 - ■»  A  :  k. 

5.  If  A]Q  \~  A  4=4  U  :  k  then  A;OhB  4=4  A  :  k. 

6.  If  A;  O  b  K  < 4  L  :  kind~  then  A;  ft  b  L  4 - 4  K  :  kind  . 

Proof:  By  simultaneous  induction  on  the  given  derivation.  □ 

Theorem  20  (Transitivity  of  algorithmic  equality) 

1.  If  A;  Q  b  oi  4 - >  <j 2  :  <f>  and  A;  ft  b  o2  4 - >  <r3  :  <f>  then  A;  Vt  b  ay  4 - >  0-3:$. 


by  Si 
by  inversion 
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2.  If  A;  b  M  N  :  r  and  A;0  h  JV  O  :  r  then  A;  fi  h  M  O  :  r. 

3.  If  A;  b  M  <■ - »  N  :  r  and  N  < >  O  :  r  f/jen  A;  fi  h  A/  <■ - ■>  O  :  r. 

If  A;  h  A  -<=^-  H  :  ft  and  A;  0  I—  B  -<=>-  C  :  ft  then  A;  h  A  -<=>-  C  :  ft. 

5.  //A;fihj4  4 - >  B  :  ft  and  A;  O  I—  A?  * ■>  C  :  ft  fhen  A;  h  A  * - ■»  C  :  ft. 

h.  If  A;  h  A'  -<=>-  A  :  kincH  and  A;  b  L  -<=>-  A'  :  kind 

then  A;  0  h-  K  <£=>-  L'  :  kindA 

Proof:  By  simultaneous  inductions  on  the  structure  of  the  given  derivations.  In  each 
case,  one  of  the  two  derivations  is  strictly  smaller,  while  the  other  derivation  is  either 
smaller  or  the  same.  The  proof  requires  determinacy  and  follows  the  proof  in  [29].  □ 

2.9  Completeness  of  algorithmic  equality 

We  now  develop  the  completeness  theorem  for  the  type-directed  equality  algorithm  by 
an  argument  via  logical  relations.  The  logical  relations  are  defined  inductively  on  the 
approximate  type  of  an  object. 

The  completeness  theorem  can  be  stated  as  follows  for  type-directed  equality: 

If  A;  T  b  M  =  N  :  A  then  A";  T”  h  M  N  :  A~. 

Here  we  define  a  logical  relation  A;  Q  b  M  —  N  e  [t]  that  provides  a  stronger 
induction  hypothesis  s.t. 

1.  if  A;  T  b  M  =  N  :  A  then  A";  T"  h  M  =  Ne  [A~j 

2.  if  A;  T  h  M  =  N  G  [A~]  then  A";  hhl  N  :  A~ 

Following  Harper  and  Pfenning,  we  define  a  Kripke  logical  relation  inductively  on 
simple  types.  At  base  type  we  require  the  property  we  eventually  want  to  prove.  At 
higher  types  we  reduce  the  property  to  those  for  simpler  types.  We  extend  here  the 
logical  relation  given  in  [29]  to  allow  modal  contexts.  Since  we  do  not  introduce  any 
new  modal  variables  in  the  algorithm  for  type-directed  equivalence,  the  modal  context 
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does  not  change  and  is  essentially  just  carried  along.  In  contrast,  the  context  hi  for 
bound  variables  may  be  extended.  This  is  accounted  for  in  the  case  for  the  function 
type.  We  say  IV  extends  hi  (written  hi'  >  hi)  if  hi'  contains  all  declarations  in  hi  and 
possibly  more. 

1.  A;!2bcr  =  #£[-]iff£r=-  and  9  — 

2.  A;  12  b  a  =  9  £  [<£>,  x:r\  iff  a  —  (o',  M/x)  and  9  =  ( 9 N/x)  where 
A;  Oh  o'  =  9’  E  [$]  and  A;OhM  =  A£  [r], 

3.  A;OhM  =  iV£  [a]  iff  A;  12  b  M  N  :  a 

4.  A;  12  b  M  —  N  £  [ti  — >•  r2]  iff  for  every  12'  extending  12  and  for  all  M\  and 
s.  t.  A;  12'  b  Mi  =  JV[£  [ti]  we  have  A;  12'  b  M  Mi  —  N  Ni  £  [r2]. 

5.  A;  12  b  A  =  B  £  [type-]  iff  A;  12  b  A  B  :  type- 

6.  A;  12  b  A  =  B  £  [r  — >  a]  iff  for  every  12'  extending  12  and  for  all  M  and  N 
s.  t.  A;  12'  b  M  =  N  £  [r]  we  have  A;  12'  b  A  M  —  B  N  £  |/c]. 

The  general  structural  properties  of  logical  relations  that  we  can  show  directly  by 
induction  are  exchange,  weakening,  contraction  and  strengthening.  We  only  prove  and 
use  weakening. 

Lemma  21  (Weakening) 

For  all  logical  relations  R,  if  A;  (12, 12')  b  R  then  A;  (12,  x:r,  12')  b  R. 

Proof:  By  induction  on  the  structure  of  the  definition  of  R.  □ 

It  is  straightforward  to  show  that  logically  related  terms  are  considered  identical 
by  the  algorithm. 

Theorem  22  (Logically  related  terms  are  algorithmically  equal) 

1.  If  A;  12  b  a  =  o'  £  [$]  then  A;  12  b  a  * ■>  o'  :  $. 

2.  If  A;  12  b  M  =  N  £  [r]  then  A;  12  b  M  «  N  :  t. 
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3.  If  A;  n  b  M  < — >  N  :  r  then  A;  ft  b  M  =  N  e  [r] 

4.  If  A;  12  b  A  —  B  e  [/c]  i/ien  A;  12  b  A  •<=>■  B  \  n. 

5.  //A;  hi  4 - ■>  B  :  k  then  A;Ohi  =  Be  [/c] . 


Proof:  The  statements  are  proven  by  simultaneous  induction  on  the  structure  of  r  or 
<f>  respectively. 

Proof  of  (1):  Induction  on  <f>. 


Case  $  =  • 

A;  sm-  =  •£[•] 

A;  O  h  ■  =  ■  :  ■ 

Case  $  =  x:t 

A;  12  b  cti  =  cr2  G  [$',  x:r] 

(T 1  =  ( a,M/x )  and  cr2  =  ( a',N/x ) 

A;Oh(r  =  de  [$'] 

A;  12  b  M  =  N  e  [r] 

A;  12  b  M  <*=*►  AT  :  r 
A;  12  b  u  * - ■>  a '  : 

A;  12  b  (cr,M/x)  * - >  (a',N/x)  :  (&,x:r) 

Proof  of  (2)  by  induction  on  the  structure  of  r: 

Case  r  =  a 

A;  12  b  M  =  N  e  [a] 

A;  12  b  M  N  :  a 

Case  r  =  Ti  — >  T2 

A;  12  b  M  =  N  e  In  ->  r2] 

A;  12,  x:ti  b  x  <■ - >  x  :  Ti 

A;  12, x:T!  hx  =  i6  [ri] 

A;  Q,x:ti  hMx  =  JVa:6  [t2] 

A;  12,x:ti  b  M x  N  x  :  t2 
A;  12  b  M  TV  :  n  t2 


by  assumption 
by  rule 


by  assumption 
by  definition 


by  i.h.  (2) 
by  i.h.  (1) 
by  rule 


by  assumption 
by  definition 


by  assumption 
by  rule 
by  i.h.  (3) 
by  definition 
by  i.h.  (2) 
by  rule 
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Proof  of  (3): 

Case  t  =  a 

4 - >  TV  :  a 

<*=*  TV  :  a 
A;  Q  h  M  =  N  e  [a] 

Case  r  =  T\  — >  T2 

A;  h  M  < - *  TV  :  n  ->  t2 

A;  f2+  h  Mi  =  Ai  e  [n]  for  any  arbitrary  Q+  >  f2 
A;  12+  h  Mi  Ni  :  n 

A;  S]+hM« - >  TV  :  n  ->  r2 

A;  Q+  h  M  Mi  * - >  NNi  :  r2 

A;  Q+  h  M  Mi  =  NNi  e  [r2] 

A;  0  h  M  =  A  e  [r!  ->  r2] 


by  assumption 
by  rule 
by  def. 


by  assumption 
by  new  assumption 
by  i.h.  (2) 
by  weakening 
by  rule 
by  i.h.  (3) 
by  definition 

□ 


Lemma  23  (Closure  under  head  expansion) 

1.  If  A;  n  b  M  ^  M'  and  A;  ft  h  M'  =  N  e  [r]  then  M  =  TV  e  [r], 

2.  //A;  AT  h  TV  ^  TV'  and  A;  AT  h  M  =  TV'  e  [r]  then  M  =  TV  e  [r]. 


Proof:  By  induction  on  the  structure  of  r. 

Case  r  =  a 

A;Oh  M  ^  M' 

A;  ft  b  M'  ^  N  :a 
A;  OhM  <=*  AT  :  a 
A;  ft  b  M  =  N  e  [a] 

Case  r  =  Ti  — >  r2 
A;  AT  b  M  ^  M' 

A;  ft  b  M'  =  TV  e  [n  r2] 

A;  ft+  b  Mi  =  Ni  e  [ti]  for  ft+  >  ft 
A;  ft+  b  M'  Mi  =  NNie  [r2] 


by  assumption 
by  definition  of  [a] 
by  rule 
by  definition 


by  assumption 
by  assumption 
by  new  assumption 
definition 
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A;  b  M  Mx  ^  M'  M1  by  whr  rule 

A;  fl+  b  M  Mi  =  NN-iE.  [t2]  by  i.h. 

A;  f]  b  M  =  N  E  [ri  — ■>  t2]  by  definition  [ti  — >  r2] 

□ 

Note  that  the  proof  of  this  lemma  only  depends  on  the  structure  of  r  and  not  on 
the  derivations  of  weak  head  reduction.  This  lemma  is  critical  in  order  to  prove  that 
definitional  equal  terms  are  logically  related  under  substitutions. 

Lemma  24  (Symmetry  of  logical  relations) 

1.  //  A;  b  a  =  9  E  [<f>]  then  A;  0  b  9  =  a  E  [$] 

2.  If  A;  ft  b  M  =  N  E  [t]  then  A;flbiV  =  ¥e[r) 

3.  If  A;  b  A  =  B  E  [/c]  then  A;  0  b  B  =  A  E  [/c] 

Proof:  By  induction  on  the  structure  of  r  and  <f>,  using  lemma  on  symmetry  of  algo¬ 
rithmic  equality.  □ 

Lemma  25  (Transitivity  of  logical  relations) 

1.  If  A;  D,  b  a  =  0  E  [$]  and  A;  b  0  =  p  E  [$]  then  A;0  b  a  —  p  E  [$] 

2.  If  A;  D,  b  M  =  N  E  [r]  and  N  =  O  E  [r]  then  A;Ob  M  —  O  e\t\ 

3.  If  A;  b  A  =  B  E  [/c]  and  A;  ft  b  B  =  C  E  [ft]  then  A;  It  b  A  =  C  E  [r] 

Proof:  By  induction  on  the  structure  of  r  and  using  lemma  on  transitivity  of  algo¬ 
rithmic  equality.  □ 

Lemma  26  (Definitionally  equal  terms  are  logically  related  under  substitutions) 

1 .  If  A;  T  b  p  =  p'  :  T  and  A-;  It  b  cr  =  9  E  [T“] 
then  A-;f2  b  ([cr]p)  =  {[0\p')  E 

2.  If  A;  T  b  M  =  N  :  A  and  A";  b  a  =  9  E  [r-] 
then  A-;f2  b  [a\M  =  [9}N  E  [A-]. 
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3.  If  A;  r  b  A  =  B  :  K  and  A";  ft  b  a  =  9  G  [T"] 
t/ien  A-;  ft  b  [a] A  =  [0].B  G  [X-]. 

Proof:  By  induction  on  the  derivation  of  definitional  equality.  The  proof  follows  the 
development  in  [29].  We  only  give  the  case  for  modal  variables. 

©i 

Case  V  =  A2;T  b  p  =  p'  :  T 

Ai,  A2;  T  b  w[p]  =  u[p']  :  [p\A 

A^,u::T-hA-,  A^;ft  b  &  =  0  [T“] 

Ar,u::T-hA-,  A2";ft  b  ([a]p)  =  {W)  e  M 
Aj Aj;fi  b  ([a]p)  * — +  ([0]p')  :  tf" 

A]“,  Aj;  ft  b  w[[cr]p]  4 - »  -u[[0]p']  :  A~ 

Af,u::^~\-A-,  A^jft  b  u[[a]p]  =  u[[0]p']  €  [A~] 

Ab,u::T-hA-,  A^;ft  b  [a](w[p])  =  [0](u[p'])  G  [A"] 

A^,u::T-hA-,  Ajjft  b  [a](w[p])  =  [9\(u[p'})  G  [[pJbP] 

□ 


by  assumption 
by  i.h. 
by  lemma  22 
by  rule 
by  lemma  22 
by  subst.  definition 
by  erasure  lemma  16 


Lemma  27  (Identity  substitutions  are  logically  related) 

A -;T-  b  idr  =  idr  G  [T"] 

Proof:  By  structural  induction  on  [T-]  and  theorem  22(3).  □ 

Theorem  28  (Definitionally  equal  terms  are  logically  related) 

1.  If  A;T  b  M  =  N  :  A  then  A -;T~  b  M  =  iV  G  [bP], 

A  If  A;  T  b  4  =  S  :  K  then  A";  T"  b  A  =  B  G 
Proof:  Direct  by  lemma  27  and  lemma  26.  □ 

Theorem  29  (Completeness  of  algorithmic  equality) 

1.  If  A;Tb  M  =  N  :  A  then  A^T"  b  M  «  A  :  A~ . 

2.  If  A;  T  b  A  =  B  :  K  then  A~;T~  b  A  <=>  B  :  K~ . 

Proof:  Direct  by  theorem  28  and  theorem  22  □ 
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2.10  Soundness  of  algorithmic  equality 

We  prove  soundness  of  algorithmic  equality  via  subject  reduction.  The  algorithm  for 
type-directed  equality  is  not  sound  in  general.  However,  when  applied  to  valid  objects 
of  the  same  type,  in  a  valid  modal  context  A  it  is  sound  and  relates  only  equal  terms. 

Lemma  30  (Subject  reduction) 

If  A-;T-hM*  M'  and  A;  T  b  M  :  A  then  A;  T  b  M'  :  A  and  A;  T  b  M  =  M'  :  A. 

Proof:  By  induction  on  the  definition  of  weak  head  reduction.  The  two  cases  follow 
the  proof  in  [29]. 

Case  W  = - 

A-;T"  b  (A x:A1.M2)  M1  ^  [id T-,M1/x\M2 


A;  T  b  (A x:Al.M2)  M1  :  A 
A;  T  b  Xx\A\  ,M2  :  Tlx:Bi.B2 

A;  T  b  Mi  :  Bi  and  A;  T  b  [idp,  Mi/x]B2  =  A  :  type 
A;  T  b  Ai  :  type  and  A;  T,  x\Ai  b  M2  :  A2 
A;  T  b  Y[x:A\.A2  =  Yix\Bi.B2  :  type 


by  assumption 
by  inversion  lemma  12 

by  inversion  lemma  12 


A;  T  b  A\  =  B\  :  type  by  injectivity  of  products  (lemma  15) 

A;  T,  x\Ai  b  A2  =  B2  :  type 


A;  T  b  Mi  :  A1 
A;T  b  idp  :  T 

A;  T  b  (idp,  Mi/x)  :  (T,^) 

A;  T  b  [\6t,Mi/x]A2  =  [idr,Mi/x\B2  :  type 
A;  T  b  [idp,  Mi/x\A2  =  A  :  type 
A;  T,  x\Ai  b  M2  =  M2  :  A2 
A;  T  b  Mi  =  Mi  :  Hi 

A;  T  b  (A x:Ai.M2)  Mi  =  [idp,  Mi/x\M2  :  [idp,  Mi/x]A2 

A;  T  b  (A x:Ai.M2)  Mi  =  [idp,  Mxfx\M2  :  A 

A;  T  b  (A x:Ax.M2)  Mi  =  [id r-,Mx/x\M2  :  A 

A;  T  b  [idr ,Mi/x\M2  :  [id r,Mi/x\A2 

A;  T  b  [idp,  Mi/x\M2  :  A 

A;  T  b  [idp -,Mi/x\M2  :  A 


by  type  conversion 
by  definition 
by  substitution  rule 
by  substitution  lemma  6 
transitivity 
by  reflexivity 
by  reflexivity 
by  rule 
by  type  conversion 
by  erase  lemma  16 
by  subst.  property  lemma  6 
by  type  conversion 
by  erase  lemma  16 
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A-;r-  b M^AN 

Case  VV  = - 

A";  r  h  M  M2^A>N  M2 

A;  r  b  M  M2  :  A 

A;  T  h  [idr,  M2/x\A\  =  A  :  type 

A;  T  b  M  :  Tix:A2.Ai  and  A;  V  b  M2  :  A2 

A;T  b  N  :  Ux-.A^ 

A;T  b  M  =  N  :  n 'x\A^.Ax 

A;  T  b  M2  =  M2  :  A2 

A;  T  b  M  M2  =  NM2  :  [idr,  M2/x]^i 

A;  T  b  M  M2  =  N  M2  :  A 

A;  T  h  N  M2  :  [idr,  M2/x\A1 

A;  T  b  N  M2  :  A 


by  assumption 
by  inversion  lemma  12 

by  i.h. 
by  i.h. 

by  reflexivity 
by  rule 
by  type  conversion 
by  rule 
by  type  conversion 

□ 


Theorem  31  (Soundness  of  algorithmic  equality) 

1.  If  A;  T  b  o  :  ^  and  A;  T  b  o'  :  ^  and  A-;  b  o  * - »  o'  :  \I/_  then 

A;  r  b  o  =  o'  : 

A  If  A;  T  b  M  :  A  and  A;  T  b  A  :  A  and  A";  T"  b  M  «  N  :  then 

A;  T  b  M  =  N  :  A 

5.  If  A;  T  b  M  :  A  and  A]T  \~  N  :  B  and  A-;  T-  b  M  * - >  A  :  r  then 

A;  T  b  M  =  N  :  A  and  A;T  b  d  e  B  :  type  and  A~  =  B~  =  r. 

4.  If  A;  T  b  A  :  K  and  A;  T  b  5  :  K  and  AyTbd  4=^  B  :  K~  then 
A;Tb  A  =  B  :  A. 

5.  If  A;  T  b  A  :  A'  and  A;T  b  B  :  L  and  A-;  r~  b  d  4 - >  A  :  n  then 

A;  T  b  d  =  B  :  K  and  A;  T  b  K  =  L  :  kind  and  K~  —  L~  —  k. 

6.  If  A;  T  b  K  :  kind  and  A;  T  b  L  :  kind  and  A-;  b  K  L  :  ki nd  then 
A;T  b  K  =  L  :  kind. 
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Proof:  By  induction  on  the  structure  of  the  given  derivations  for  algorithmic  equality. 
Proof  for  (1). 

Case  T  = _ 

A-,T~  b  •  < — 

A;  T  h  •  =  •  :  •  by  rule 


Case  T  = 


A”;r~  b  M 


N  :  A~ 


A~;T~  b  ( a,M/x ) 


A  ;  T  b  a  < - >  o'  :  d' 

( o',N/x )  : 


A;  r  h  (cr,  M/x)  :  (V,x:A) 

A;  r  b  a  :  ^  and  A;  T  b  M  :  [a]A 
A;  r  b  (a',N/x)  :  (V,x:A) 

A;  T  b  tr  :  'P  and  A;  T  b  A  :  [a']bl 
A;rbfrEff':$ 

A;  T  b  A  :  type 

A;  r  b  [a] A  =  [a7] A  :  type 

A;  T  b  [a7] A  =  [a]A  :  type 

A;T  b  N  :  [a] A 

A -;r-  blfyyiV:  ([cr]A)“ 

A;  T  b  M  =  N  :  [cr]A 

A;  r  b  (cr,  M/x)  =  (cr7,  A/x)  :  (^,  x: A) 


by  assumption 
by  inversion 
by  assumption 
by  inversion 
by  i.h. 

by  validity  of  ctx  d' 
by  functionality  lemma  9 
by  symmetry 
by  type  conversion 
by  erase  lemma  16 
by  i.h. 
by  rule 


A;  T  b  A  :  A 
A;  T  b  M  :  A 
A;  T  b  A  =  M'  :  A 
A;  T  b  M'  :  A 


by  assumption 
by  assumption 
by  subject  reduction  on  Wi 
by  validity 
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A;  T  b  M'  =  M  :  A 
A;  T  b  N  =  M  :  A 


Case  T  = 


Wi 

A“;  r~  b  M  ^  M' 


AAr”  b  N 


r2 


M'  :  A~ 


Case  T  = 


A-;r~  b  N  M  :  A~ 
similar  to  the  one  above  (symmetric). 

A-;  b  M  < - >  N  :  A~ 


A-;  b  M  N  :  A~ 


A;T  b  M  :  A 
A;T  b  N  :  A 
A;  T  b  M  =  N  :  A 


Case  T 


Ti 

A-;  r~,x:Ti  b  M x  N x  :  r2 
A";  T"  b  M  «  N  :  n  ->  r2 


by  i.h.  on  T2 
by  transitivity 


by  assumption 
by  assumption 
by  i.h. 


A;  r  b  M  :  n x:A1.A2 
A;ThN  :  UxiA^ 

(Ai)~  =  Ti  and  (A2)~  =  r2 
A;  T  b  Yix:A1.A2  :  type 
A;  T  b  Ai  :  type  and  A;  T,  x\Ai  b  A2  :  type 
A;  T,  a;:Ai  b  x  :  A\ 

A;  T,  x:A\  b  M  :  Ihr:bli.bL2 
A;  T,  x\A\  b  M  x  :  A2 
A;  T,  x:Ai  b  N  x  :  A2 
A;  T,  a;:Ai  b  M  x  =  N x  :  A2 
A;Tb  M  =  N  :  Ihr:Ai.A2 


by  assumption 
by  assumption 
by  erasure  def. 
by  validity 
by  inversion  lemma  12 
by  var  rule 
by  weakening 
by  rule  (app) 
by  rule  (app)  and  weakening 
by  i.h.  on  Ti 
by  rule  (extensionality) 


Proof  (3),  we  only  give  the  case  for  modal  variable. 


Case  S 


<Si 

u::('h)_  b  (Ai)~  e  A~  (A)-;  r~  b  <7  < - *  o'  :  if- 

(A)-;  r~  b  u[o\  < - >  u[o']  :  r 


53 


CHAPTER  2.  DEPENDENTLY  TYPED  LAMBDA  CALCULUS  BASED  ON  MODAL  TYPE  THEORY 


A;  T  b  u[cr\  :  A 

by  assumption 

A;  T  b  u[a']  :  B 

by  assumption 

A;  T  b  A  =  [ujAx  :  type  and  A  =  A1;  -ucikhA!,  A2 

A;  T  b  cr  :  4/ 

by  inversion  lemma  12 

Ai,  u::ty\-Ai,  A2;  T  b  B  =  [er'JAi  :  type 

A;  T  b  cr'  :  4/ 

by  inversion  lemma  12 

A1,w::'l/hA1,  A2;  T  b  [cr^Ai  =  1?  :  type 

by  symmetry 

Ai,  u::4/hAi,  A2;  T  b  <r  =  o'  :  d' 

by  i.h.  on  <Si 

Ai,  u::4/hAi,  A2;  d'  b  A\  :  type 

by  validity  and  weakening 

Ai,  u::4/hAi,  A2;  d'  b  A\  =  Ai  :  type 

by  reflexivity 

A1;  ■u::vkhA1,  A2;  T  b  [cr]2li  =  [a7]  A],  :  type 

by  functionality  lemma  9 

A1;  A2;  T  b  A  =  [cr'JA!  :  type 

by  transitivity 

Ai,  A2;  T  b  A  =  H  :  type 

by  transitivity 

Ai,  uc'I'hAi,  A2;  T  b  u[a\  =  u[a']  :  [cr]Ai 

by  rule 

A1;  ■u::'khA1,  A2;  Y  b  u[a]  =  u[a']  :  A 

by  type  conversion 

(A)- =  ^A,)- =  (A,)- =  r 

by  erasure  property 

{BY  =  =  (Ai)'  =  r 

by  erasure  property 

□ 


Corollary  32  (Logically  related  terms  are  definitionally  equal) 

1.  If  A;T  b  M  :  A,  A;T  b  N  :  A,  and  A";r“  b  M  =  N  e  [A~j 
then  A;  T  b  M  =  N  :  A. 

2.  If  A;  T  b  A  :  K,  A;  T  b  B  :  K,  and  A";  T"  b  A  =  B  G  [K~] 
then  A;  T  b  A  =  B  :  K. 

Proof:  Direct  from  assumptions  and  prior  theorems. 

A-;  r~  b  M  =  N  e  [A-]  by  assumption 

A-;  r~  b  M  N  :  A~  by  completeness  theorem  22 

A;  r  b  M  =  N  :  A  by  soundness  theorem  31 

□ 
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We  can  now  show  that  the  inference  rules  for  algorithmic  equality  constitute  a 
decision  procedure.  We  say  an  object  is  normalizing  iff  it  is  related  to  some  term  by 
the  type-directed  equivalence  algorithm.  More  precisely,  M  is  normalizing  at  simple 
type  t  iff  A;  Q  b  M  M'  :  r  for  some  term  M' .  By  transitivity  and  symmetry,  this 
implies  that  A;  b  M  -<=>-  M  :  t.  A  term  M  is  structurally  normalizing  iff  it  is  related 

to  some  term  by  the  structural  equivalence  algorithm,  i.e.  A;  b  M  < - »  M'  :  t  for 

some  M' .  Equality  is  decidable  on  normalizing  terms. 

Lemma  33  (Decidability  for  normalizing  terms) 

1.  If  A;  Q  b  cr  < - -»  a'  :  <f>  and  A;  Q  b  r  < - ■>  t'  :  <f> 

then  it  is  decidable  whether  A;  b  a  < — *  r  :  <f>. 

2.  //A;flbtf^tf':T  and  A;HbiV  N'  :  r 
then  it  is  decidable  whether  A;  b  M  •<=>  N  :  r. 

3.  //A;ObM« — >  M'  :  n  and  A;ObiV  ♦ — >  JV'  :  r2 

then  it  is  decidable  whether  A;  b  M  < — >  N  :  r3  for  some  r3. 

If  A;  b  A  A'  :  k  and  A;  L2  b  B  B'  :  k 

then  it  is  decidable  whether  A;  b  A  B  :  k. 

5.  If  A;  b  A  4 — ■»  A'  :  K\  and  A;  b  B  < — >  B'  :  K2 

then  it  is  decidable  whether  A;  b  A  <■ — >  B  :  n3  for  some  k3. 

6.  If  A;  H  b  K  K'  :  ki nd  and  A;  H  b  L  4=^  L'  :  ki nd 
then  it  is  decidable  whether  A;  b  K  L  :  kincE. 

Proof:  By  structural  induction  on  the  derivations  □ 

Theorem  34  (Decidability  of  algorithmic  equality) 

1.  If  A;  T  b  M  :  A  and  A;  T  b  iV  ;  A 

then  it  is  decidable  whether  A-;  LbM  4=>-  N  :  A-. 

A  If  A;  T  b  A  :  Jl  and  A;  T  b  B  :  K 

then  it  is  decidable  whether  A-;  f"  b  A  4=4-  B  :  K~ . 
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3.  If  A;  r  b  K  :  kind  and  A;  T  b  L  :  kind 

then  it  is  decidable  whether  A-;  T~  b  K  L  :  kind  . 

Proof:  By  reflexivity  of  definitional  equality  and  completeness  of  algorithmic  equality, 
both  M  and  N  are  normalizing.  Therefore,  by  the  previous  lemma,  the  algorithmic 
equivalence  is  decidable.  □ 

Theorem  35  (Decidability  of  definitional  equality) 

1.  If  A;  T  b  M  :  A  and  A;TbiV:d  then  it  is  decidable  whether  A]  T  b  M  =  N  :  A. 

2.  If  A;  T  b  A  :  K  and  A;  T  b  B  :  K  then  it  is  decidable  whether  A;  T  b  A  =  B  :  K . 

3.  If  A;T  b  K  :  kind  and  A;  T  b  L  :  kind  then  it  is  decidable  whether 
A;Tb  K  =  L  :  kind. 

Proof:  By  soundness  and  completeness  it  suffices  to  check  algorithmic  equality  which 
is  decidable  by  the  previous  lemma.  □ 

2.11  Decidability  of  type-checking 

Next,  we  give  a  type-checking  algorithm  which  uses  algorithmic  equality.  It  is  bi¬ 
directional  and  uses  the  following  two  judgments: 

A;  T  b  a  T  Check  a  against  T 

A;  T  b  M  A  Synthesize  a  type  A  for  M 

While  we  can  synthesize  a  type  A  for  an  object  M,  we  need  to  check  the  substitution 
a  against  a  context  T.  We  assume  that  the  modal  context  A  and  the  context  T  and 
T  are  valid. 

Substitutions 

A;  T  b  or  <*=  T  A;  T  b  M  =>  A'  A;  T  b  A1  [a\A 
A;T  b  •  <*=  •  A;  T  b  (a,M/x)  ^  {^f,x:A) 

Note  that  we  synthesize  a  type  A1  for  M,  and  then  check  separately  that  A'  is 
definitional  equal  to  [cr]A 
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Objects 

x:A  in  Y  c:A  in  S  A,  u::(VhA),  A';  V  b  a  4=  V 

A;T  h  x  A  A;T  h  i  A;  m::(\&kA),  A';  T  b  u[a]  =>•  [cr]^4 

A;  T  b  A\  type  A;  T,  x\A\  b  M  =>•  A2 
A;  T  b  A x:A\.  M  ILe:Al.  A2 

A;  r  b  Mi  II:c:A.2.  Ai  A;  T  b  M2  =>  A'2  A;  T  b  A'2  •<=>■  A2  :  type 
A;  r  b  Ml  M2  =»  [idr,  M2/x\A1 

Families 

a  =>■  K  in  signature  A;  Y  b  A\  =>■  type  A;  Y,x\Ai  b  A2  =>■  type 
A;  T  b  a .  =>■  K  A;  T  b  flaiiA!.  A2  =>•  type 

A;  T  b  A  =>  n  x:B'.  K  AjTbM^B  A;  T  b  B  B'  :  type 
A;T  b  AM  [idr,  M/x]K 

Kinds 

A;  r,  x:A  b  K  =>  kind  A;  T  b  A  =>■  type 
A;  T  b  type  =>•  kind  A;T  b  fhr:A  K  =>■  kind 

Similar  rules  exist  for  checking  validity  of  context  and  signatures. 

Theorem  36  (Correctness  of  algorithmic  type-checking) 

1.  If  A;  T  b  o  4=  d'  then  A;  Y  b  o  :  Ac 

2.  If  A;  T  b  M  =>  A  then  A;  T  b  M  :  A. 

3.  If  A;  T  b  a  :  if  then  A;  T  b  cr  -4=  d/. 

A  If  A;  T  b  M  :  A  then  A;  T  b  M  ^  A'  for  some  AI  such  that  A;  T  b  A  =  A'  :  type. 

Proof:  Part  (1)  and  (2)  follows  by  simultanous  induction  on  the  first  derivation  using 

validity,  soundness  of  algorithmic  equality  and  the  rule  of  type  conversion.  Part  (3)  and 
(4)  follows  by  induction  on  the  first  derivation  using  transitivity  of  equality,  inversion 
on  type  equality  and  completeness  of  algorithmic  equality.  □ 
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Since  the  algorithmic  typing  rules  are  syntax-directed  and  algorithmic  equality  is 
decidable,  there  either  exists  a  unique  A'  s.t.  A;  T  b  M  =>■  A'  or  there  is  no  such  A1 .  By 
correctness  of  algorithmic  type-checking,  we  have  A;  T  b  M  :  A  iff  A;  T  b  A'  =  A  :  type 
which  can  be  decided  by  checking  A~;  T~  b  A'  A  :  type. 

Theorem  37  (Decidability  of  type-checking) 

1.  It  is  decidable  if  A  is  valid. 

2.  Given  a  valid  A,  it  is  decidable  ifT  is  valid. 

3.  Given  a  valid  A,  T,  M  and  A,  it  is  decidable  whether  A;  T  b  M  :  A. 

4-  Given  a  valid  A,  T,  A  and  K ,  it  is  decidable  whether  A;  T  b  A  :  K. 

5.  Given  a  valid  A,  T,  and  K,  it  is  decidable  whether  A;T  b  K  :  kind. 

Proof:  We  note  that  the  algorithmic  typing  rules  are  syntax-directed  and  algorithmic 
equality  is  decidable  (see  theorem  34).  Hence,  there  either  exists  a  unique  A'  s.t. 
A;  r  b  M  =>  A'  or  there  is  no  such  A' .  By  correctness  of  algorithmic  type-checking, 
we  then  have  A;T  b  M  :  A  iff  A;  TA'  =  A  :  type,  which  can  be  decided  by  checking 
A“;  r~  b  A1  A  :  type.  □ 

The  correctness  of  algorithmic  type-checking  allows  us  to  show  strengthening. 

Lemma  38  (Strengthening) 

1.  If  A;  T  ,x:A,  T'  b  J  and  x  does  not  occur  in  the  free  ordinary  variables  of  J  and 
does  not  occur  in  the  free  ordinary  variables  ofTr,  then  A;  T,  T'  b  J. 

2.  If  A,  u::(Ti-H),  A';  T  b  J  and  u  does  not  occur  in  the  free  modal  variables  of  J 
and  u  does  not  occur  in  the  free  modal  variables  of  A', 

then  A,  A';  T  b  J. 

Proof:  Strengthening  for  the  algorithmic  version  of  type-checking  follows  by  structural 
induction  over  the  structure  of  the  given  derivation,  taking  advantage  of  the  obvious 
strengthening  for  algorithmic  equality.  Strengthening  for  the  original  typing  rules  then 
follows  by  soundness  and  completeness  of  algorithmic  typing.  Strengtheing  for  equal¬ 
ity  judgments  follows  from  completeness  (theorem  29,  soundness  (theorem  31),  and 
strengthening  for  the  typing  judgments.  □ 
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2.12  Canonical  forms 

In  this  section,  we  discuss  canonical  forms  where  canonical  forms  are  //-long  and  j3- 
normal  forms.  We  will  start  by  instrumenting  the  algorithmic  equality  judgements  to 
produce  the  mediating  object  for  two  equal  objects.  These  mediating  objects  will  be 
unique.  Since  algorithmic  equality  uses  simplified  types,  no  typing  information  is  avail¬ 
able  on  the  bound  variables  in  lambda-abstractions.  Fortunately,  we  can  reconstruct 
the  typing  information  and  the  type  labels  are  uniquely  determined. 

We  formalize  this  idea  using  quasi-canonical  and  quasi-atomic  forms,  in  which  type 
labels  have  been  deleted. 

Quasi-canonical  Objects  U  ::=  Xx.  U  \  R 

Quasi-atomic  Objects  R  ::=  c\x\  u[rj\  \  RU 
Quasi-canonical  Substitutions  rj  ::=  -\r],U/x 

Now,  we  instrument  the  algorithmic  equality  relations  to  extract  a  common  quasi- 
canonical  or  quasi-atomic  form  for  the  terms  being  compared. 


Instrumented  type-directed  object  equality 

A;OhM*  M'  A;Oh  M'  «  N  :  R 

A;  Oh  M  ^  N  :  at  R 

A;  O  b  N  ^  N'  A;  O  b  M  N'  :  a  ft  R 
A;  O  b  Ad  <==>  N  :  a  ft  R~ 

A;  Ob  M< — >N:alR  A;  O,  x:n  b  Mx  <=>  Nx  :  r2  ft  U 

A;  O  b  M  ^  N  :ai\  R  A;  O  b  M  N  :  n  ->•  r2  ft  A  x.U 

Instrumented  structural  object  equality 

_ x:t  in  O _  c:r  in  signatures 

A;  O  b  x  * - >  x  :  t  l  x  A;  O  b  c  < - >  c  :  r  j  c 

«::($  b  r)  in  A  A;  O  b  o  « - >  o'  :  $  ft  rj 

A;  O  b  u[a]  < — 4  UW]  '■  T  i  u[v\ 
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A;  n  b  M1  * — »  Nt  :  r2  ->  n  I  R  A;  Q  b  M2  «  N2  ;  r2  ff  U 
A;  fi  b  Afi  M2  < - >  A,  iV2  :  t,  f  AH 


Instrumented  substitution  equality 

A;fiha  4 - ■»  cb  :  $  tT  77  A;0  h  M  4=4>  N  :  r  ft  U 

A;Oh-  4 - 4  •  :  •  ft  •  A;  Q  b  (a,  M/x)  4 - *  (cb,  IV/x)  :  (<&,  x:t)  ft  (77,  U /x) 

We  say  that  a  term  M  has  a  quasi-canonical  form  U  if  A;  Q  b  M  •<=>■  M  :  t  ft  U 
for  appropriate  A,  fl  and  r,  and  similarly  for  quasi-canonical  forms.  From  the  previous 
development,  it  follows  that  every  well-formed  term  has  a  unique  quasi-canonical  form. 

Theorem  39  (Quasi-canonical  and  quasi-atomic  forms) 

1.  If  A;T  b  M1  :  A  and  A;  T  b  M2  :  A  and  A-jT"  b  Mx  «  M2  :  A~  ff  U 
then  there  is  an  N  such  that  \ N\  =  U ,  A;  T  b  N  :  A,  A;  T  b  Mi  =  N  :  A  and 
A;  r  b  M2  =  AT  :  A. 

A  If  A;  T  b  M !  :  A  ond  A;  T  b  M2  :  A  and  A-;  T~  b  M{  < - 4  M2  :  A”  f  R  then 

A;  T  b  Ax  =  A2  :  type,  A~  =  B~  =  r  and  there  exists  a  N  such  that  |iV|  =  R, 

A;  T  b  N  :  A,  A;  T  b  M1  =  N  :  A  and  A;  T  b  M2  =  A  :  A 

5.  If  A; T  b  <Ti  :  a?zd  A;T  b  tr2  :  $  and  A“;T~  b  a\  * - 4  <j2  :  ft  77  i/ien 

tdere  exists  a  r  stic/i  that  |r|  =  77,  A;  T  b  r  :  T,  A;  T  b  ay  =  r  :  and 

A;  T  b  a2  =  r  : 


Proof:  By  simultaneous  induction  on  the  instrumented  equality  derivations,  using 
validity  of  T,  functionality  lemma  9,  type  conversion  and  symmetry  for  the  substitution 
case  (part  3).  □ 

In  the  implementation,  we  use  a  stronger  normal  form  where  existential  variables 
(represented  here  by  modal  variables)  must  also  be  of  atomic  type.  This  is  accomplished 
by  a  technique  called  lowering.  Lowering  replaces  a  variable  u::(^/i-I1x:A1.A2)  by  a  new 
variable  7//:('F,  x:A1i-A2).  This  process  is  repeated  until  all  existential  variables  have 
a  type  of  the  form  T  b  h  N\  . . .  A&.  This  operation  has  been  proved  correct  for  the 
simply- typed  case  by  Dowek  et  al.  [22],  but  remains  somewhat  mysterious.  Here,  it  is 
justified  by  the  modal  substitution  principle. 
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Lemma  40 

1.  (Lowering)  If  A,  u::(^f\-Ux:Ai.  A2),  A' ]T  \~  M  :A 
then  A,u,::^>,x:A^A2),A'*]Y*  b  M *  :  A* 
where  ( P )*  =  l(Xx:Ai.u'[\d^,x/x])/ulP. 

2.  (Raising)  If  A,  u'vf'Y,  x:Ai\-A2),  A';  T  b  M  :  A 
then  A,u::(MLr:,4i.  A2),  A/+;r+  b  M+  :  A+ 
where  ( P)+  =  |(«[id^]  x)/u'\P. 

3.  ()+  and  ()#  are  inverse  substitutions  (modulo  definitional  equality). 

Proof:  Direct,  by  weakening  and  the  modal  substitution  principle.  For  part  (1)  we 
observe  that  A,  u/::('F,  x\A\\-A2)\  b  \x\Ai.u'[\d^}  x/x]  :  Iix:A\.  A2. 

For  part  (2)  we  use  instead  that  A,  ■u::('Fi-na;:bLi.  A2 );  ,x\Ai  b  ufidvj,]  x  :  A2.  Part  (3) 
is  direct  by  calculation.  □ 


Since  we  can  lower  all  modal  variables,  we  can  change  the  syntax  of  canonical  forms 
so  that  terms  u[rj\  are  also  canonical  objects  of  base  type,  rather  than  atomic  objects. 
This  is,  in  fact,  what  we  chose  in  the  implementation. 

For  deciding  definitional  equality  between  canonical  objects,  we  can  use  syntactic 
equality  modulo  renaming  of  bound  variables,  as  usual.  For  syntactic  equality  between 
two  canonical  objects  U\  and  U2,  we  write  U\  =  U2.  The  algorithm  is  similar  to  the 
structural  equality  for  objects  given  earlier,  except  we  are  now  allowed  to  descend 
into  the  body  of  A-abstractions  and  compare  the  bodies  directly.  Since  we  know,  the 
objects  are  in  canonical  form,  we  eliminate  the  weak  head  reduction  rules  and  the  eta- 
rule.  In  other  words,  type-directed  equality  can  be  replaced  by  syntactic  equality  for 
A-abstractions. 

Finally,  we  would  like  to  point  out  that  for  canonical  forms  we  can  refine  the  bi¬ 
directional  type  checking  rules  given  earlier  on  page  56  in  such  a  way  that  we  check 
quasi-canonical  (normal)  terms  agains  a  type  A  while  we  infer  a  type  A  for  quasi-atomic 
(neutral)  terms. 
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2.13  Implementation  of  existential  variables 

In  the  implementation  the  modal  variables  in  A  are  used  to  represent  existential  vari¬ 
ables  (also  known  as  met  a- variables),  while  the  variables  in  T  are  universal  variables 
(also  known  as  parameters). 

Existential  variables  are  created  in  an  ambient  context  and  then  lowered.  We  do 
not  explicitly  maintain  a  context  A  of  these  existential  variables,  but  it  is  important 
that  a  proper  order  for  them  exists.  Existential  variables  are  created  with  a  mutable 
reference,  which  is  updated  with  an  assignment  when  we  need  to  carry  out  a  substitu¬ 
tion  [M/u], 

In  certain  operations,  and  particularly  after  type  reconstruction,  we  need  to  abstract 
over  the  existential  variables  in  a  term.  Since  the  LF  type  theory  provides  no  means 
to  quantify  over  u::('I'i-A)  we  raise  such  variables  until  they  have  the  form  u'::(-\-A'). 
It  turns  out  that  in  the  context  of  type  reconstruction  we  can  now  quantify  over  them 
as  ordinary  variables  x':A'.  However,  this  is  not  satisfactory  as  it  requires  first  raising 
the  type  of  existential  variables  for  abstraction,  and  later  again  lowering  the  type  of 
existential  variables  during  unification  to  undo  the  effect  of  raising.  To  efficiently  treat 
existential  variables,  we  would  like  to  directly  quantify  over  modal  variables  u. 

The  judgmental  reconstruction  in  terms  of  modal  logic  suggests  two  ways  to  incor¬ 
porate  modal  variables.  One  way  is  via  a  new  quantifier  nnu::(vI'i-Hi).  A2,  the  other  is 
via  a  general  modal  operator  Proof-theoretically,  the  former  is  slightly  simpler,  so 
we  will  pursue  this  here.  The  new  operator  then  has  the  form  nDu::(Ti-H1).  A2  and  is 
defined  by  the  following  rules. 

A;  T  b  A  :  type  A,  m::(\I/kA);  T  b  B  :  type 
A;  T  b  nDw::(Tm4).  B  :  type 

A,u::(ThH);T  b  M  :  B  A;  T  b  N  :  nDw::(Tm4).  B  A;  T  b  M  :  A 

A;  T  b  A °u.  M  :  nDu::(Tm4).  B  A; f  b  N  □  M  :  [ M/ujB 

The  main  complication  of  this  extension  is  that  variables  u  can  now  be  bound  and 
substitution  must  be  capture  avoiding.  In  the  present  implementation,  this  is  handled 
by  de  Bruijn  indices.  In  this  thesis,  we  do  not  develop  a  theory  where  lambda-box  and 
pi-box  are  first  class  abstractions. 
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2.14  Conclusion  and  related  work 

In  this  chapter,  we  presented  an  abstract  view  of  existential  variables  based  on  modal 
type  theory  and  showed  that  type-checking  remains  decidable  and  canonical  forms 
exist.  Central  to  the  development  is  the  clear  distinction  between  existential  variables 
declared  in  the  modal  context  A  and  bound  variables  declared  in  the  context  T.  Unlike 
calculi  of  explicit  substitutions  [1,  22],  our  system  does  not  require  de  Bruijn  indices  nor 
does  it  require  closure  M[a ]  as  first-class  terms.  Although  the  use  of  de  Bruijn  indices  in 
calculi  of  explicit  substitutions  leads  to  a  simple  formal  system,  the  readability  may  be 
obstructed  and  critical  principles  are  obfuscated  by  the  technical  notation.  In  addition, 
some  techniques  like  pre-cooking  of  terms  [22]  and  optimizations  such  as  lowering  and 
grafting  remain  acl  hoc.  This  makes  it  more  difficult  to  transfer  these  optimizations  to 
other  calculi. 

Viewing  existential  variables  as  modal  variables  leads  to  a  simple  clean  framework 
which  allows  us  to  explain  techniques  such  as  lowering  and  raising  logically  by  substitu¬ 
tion  principles.  Surprisingly,  the  presented  framework  justifies  with  hindsight  logically 
many  techniques  and  decisions  in  the  Twelf  implementation  [53]. 
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Chapter  3 

Toward  efficient  higher-order 
pattern  unification 


Unification  lies  at  the  heart  of  automated  reasoning  systems,  logic  programming  and 
rewrite  systems.  Thus  its  performance  affects  in  a  crucial  way  the  global  efficiency 
of  each  of  these  applications.  This  need  for  efficient  unification  algorithms  has  led  to 
many  investigations  in  the  first-order  setting.  However,  the  efficient  implementation  of 
higher-order  unification,  especially  for  dependently  typed  A-calculus,  is  still  a  central 
open  problem  limiting  the  potential  impact  of  higher-order  reasoning  systems  such  as 
Twclf  [53],  Isabelle  [45],  or  AProlog  [39]. 

The  most  comprehensive  study  on  efficient  and  robust  implementation  techniques 
for  higher-order  unification  so  far  has  been  carried  out  by  Nadathur  and  colleagues 
for  the  simply-typed  A-calculus  in  the  programming  language  AProlog  [37,  38].  The 
Teyjus  compiler  [40]  embodies  many  of  the  insights  found,  in  particular  an  adequate 
representation  of  lambda  terms  and  mechanisms  to  delay  and  compose  substitutions. 
Higher-order  unification  is  implemented  via  Huet’s  algorithm  [31]  and  special  mecha¬ 
nisms  are  incorporated  into  the  WAM  instruction  set  to  support  branching  and  post¬ 
poning  unification  problems.  To  only  perform  an  occurs-check  when  necessary,  the 
compiler  distinguishes  between  the  first  occurrence  and  subsequent  occurrences  of  a 
variable  and  compiles  them  into  different  WAM  instructions.  While  for  the  first  oc¬ 
currence  of  a  variable  the  occurs-check  may  be  omitted,  full  unification  is  used  for  all 
subsequent  variables.  This  approach  seems  to  work  well  in  the  simply-typed  setting, 
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however  it  is  not  clear  how  to  generalize  it  to  dependent  types. 

In  this  chapter,  we  discuss  the  efficient  implementation  of  higher-order  pattern 
unification  for  the  dependently  typed  lambda-calculus.  Unlike  Huet’s  general  higher- 
order  unification  algorithm  which  involves  branching  and  backtracking,  higher-order 
pattern  unification  [34,  48]  is  deterministic  and  decidable.  An  important  step  toward 
the  efficient  implementation  of  higher-order  pattern  unification  was  the  development 
based  on  explicit  substitutions  and  de  Bruijn  indices  [22]  for  the  simply-typed  lambda- 
calculus.  This  allows  a  clear  distinction  between  bound  and  existential  variables  and 
reduces  the  problem  to  essentially  first-order  unification.  Although  the  use  of  de  Bruijn 
indices  leads  to  a  simple  formal  system,  the  readability  may  be  obstructed  and  critical 
principles  are  obfuscated  by  the  technical  notation.  In  addition,  as  we  have  argued 
in  Chapter  2,  some  techniques  like  pre-cooking  of  terms  and  optimizations  such  as 
lowering  and  grafting  remain  ad  hoc. 

In  this  chapter,  we  will  particularly  focus  on  one  optimization  called  linearization, 
which  eliminates  many  unnecessary  occurs-checks  and  delays  any  computationally  ex¬ 
pensive  higher-order  parts.  Based  on  the  modal  dependently  typed  lambda  calculus, 
we  present  a  higher-order  unification  algorithm  for  linear  patterns.  We  will  assume 
that  all  terms  are  in  canonical  form  and  all  existential  variables  must  be  of  atomic 
type.  As  pointed  out  previously,  this  can  be  achieved  by  lowering  and  raising  (see 
Chapter  2).  We  have  implemented  this  optimization  of  higher-order  unification  as  part 
of  the  Twclf  system.  Experimental  results  which  we  discuss  at  the  end  of  this  chap¬ 
ter  demonstrate  significant  performance  improvement,  including  those  in  the  area  of 
proof-carrying  code. 


3.1  Higher-order  pattern  unification 

In  the  following,  we  will  consider  the  pattern  fragment  of  the  modal  lambda-calculus. 
Higher-order  patterns  are  terms  where  existential  variables  must  be  applied  to  distinct 
bound  variables.  This  fragment  was  first  identified  by  Miller  [34]  for  the  simply-typed 
lambda-calculus,  and  later  extended  by  Pfenning  [48]  to  the  dependently  typed  and 
polymorphic  case.  We  enforce  that  all  terms  are  in  canonical  form,  and  the  type  of 
existential  variables  has  been  lowered  and  is  atomic.  We  call  a  normal  term  U  an 
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atomic  pattern,  if  all  the  subterms  of  the  form  u[a\  are  such  that  a  —  yi/xi, . . .  Uk/xk 
where  yi, ...  ,yk  are  distinct  bound  variables.  This  is  already  implicitly  assumed  for 
Xi,...,Xk  because  all  variables  defined  by  a  substitution  must  be  distinct.  Such  a 
substitution  is  called  a  pattern  substitution.  Moreover  the  type  of  any  occurrence  of 
u[a\  is  atomic,  i.e.,  of  the  form  a  M\  . . .  Mn.  We  will  write  Q  for  atomic  types.  As 
a  consequence,  any  instantiation  of  u  must  be  a  neutral  object  R ,  but  can  never  be 
a  lambda-abstraction.  We  write  0  for  simultaneous  substitutions  . . .  Rn/unJ 

for  existential  variables.  As  a  consequence,  canonical  forms  are  preserved  even  when 
applying  the  modal  substitution  to  instantiate  modal  variables. 

We  emphasize  that  throughout  this  chapter  we  only  deal  with  canonical  terms  de¬ 
noted  by  U  for  normal  and  R  for  neutral  objects.  Moreover,  we  only  consider  existential 
variables  of  atomic  type  and  consider  them  as  neutral  objects.  If  the  distinction  is  not 
important  we  may  still  use  M  and  N  to  denote  objects.  Similarly,  we  will  continue 
to  denote  the  declarations  of  modal  variables  in  a  modal  context  A  as  ueTi-Q,  where 
Q  —  a  M\  ...  Mn.  Where  the  distinction  is  not  important,  we  may  still  use  A  to 
denote  arbitrary  types. 

Before  we  describe  higher-order  pattern  unification,  we  give  some  details  on  modal 
substitutions.  Recall  the  definition  of  modal  substitutions  from  Chapter  2. 


A  h  6  :  A'  Modal  substitution  6  matches  context  A' 
Substitutions 


Objects 


[0](<7,t//y) 


[0i,C//ti,92](«M) 

wru) 

M(Aj/:A  U) 

Context 


(i%,  mu/y) 

c 

X 

[ie1,uiu,e2]a\u 
(i<M  (Me) 

As /:[9]A  \o\u 


I9](r,m)  =  [0]r,i:[ep 
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Typing  rules  for  modal  substitutions  are  given  next: 

Modal  substitutions 

Ab0:A'  A;[0]T  b  U  :  [0]A 
A  b  (•)  :  (•)  A  b  (07 Uju)  :  (A>::Tm4) 

We  write  idA  for  the  identity  modal  substitution  [u^id^J/u!, . . . ,  wn[id ®„]/wn]  f°r 
a  modal  context  A  =  (•,  ui::(TihAi), . . . ,  un::(^n\-An)).  By  definition,  we  then  have 
A  b  idA  :  A.  In  an  implementation,  where  existential  variables  are  realized  via  pointers, 
this  essentially  means  the  store  of  pointers  for  existential  variables  does  not  change. 
Next,  we  show  some  properties  of  modal  substitutions. 

Lemma  41  (Modal  substitution  properties) 

If  A'  b  9  :  A  and  A;TbJ  then  A';  [0]T  b  [0]  J. 

Proof:  By  simultaneous  structural  induction  on  the  second  derivation.  □ 

Similar  to  composition  of  ordinary  substitution,  we  prove  some  composition  for 
modal  substitutions. 

Lemma  42  (Composition  of  modal  substitutions) 

i-  W(PiIH)  =  [[felsjM 
*•  IWi]!/)  =  IMflJt/ 

3-  WdOiP)  =  [Wip 

4-  PsKPilA  — 

5.  wd«i]A')  = 

«.  PddOijr)  =  [W9jr 

Proof:  By  simultaneous  induction  on  the  definition  of  modal  substitutions.  □ 

In  addition,  we  show  the  following  property  relating  modal  and  ordinary  substitu¬ 
tions. 
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Lemma  43  Assume  A;  Y  b  a  :  and  A'  b  6  :  A. 

1-  M(k]r)  =  [i»W(I»]r) 

2.  M(k]tO  =  PMtMtO 

3.  [9](k]JS)  =  [[0]a]([0]fl) 

4.  [#1([<t]3)  =  PMM3) 

J.  M(MA')  =  [[0]<x]([0]A) 

Proof:  By  simultaneous  induction  on  the  definition  of  ordinary  substitutions.  □ 

Finally,  we  note  that  applying  a  modal  substitution  6  to  a  pattern  substitution  a 
does  not  change  a  itself,  since  the  range  of  a  refers  only  to  bound  variables,  while  9 
refers  to  modal  variables. 

Lemma  44 

If  A'  b  6  :  A  and  a  is  a  pattern  substitution,  s.t.  A;T  h  tr  :  $  then  [0](cr)  =  a 

Proof:  Induction  on  the  structure  of  a  □ 

Now,  we  give  the  judgments  for  describing  higher-order  pattern  unification. 

A;  T  b  U\  =  U2  /  (A',  6)  Unification  of  normal  atomic  patterns 
A;  T  lb  R\  =  R-2  /  (A',  6)  Unification  of  neutral  atomic  objects 

We  assume  U\  (resp.  Ri)  and  U2  (resp.  R2)  are  normal  atomic  patterns,  are  well- 
typed  in  the  context  V  and  have  the  same  type  A.  Since  their  type  does  not  play  a  role 
during  unification,  we  omit  it  here.  6  is  a  modal  substitution  for  the  modal  variables 
in  A  s.t. 


A'  b  6*  :  A  and  A';  [0]r  b  [9]Ui  =  [6\U2  :  {9}A 


Since  we  require  that  all  existential  variables  have  atomic  type  the  modal  context 
A  has  the  following  form: 


A  U\ .  .\EUh(5:l ,  ■  ■  ■  ■  Un.  .H/ n^Qn 
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Moreover,  we  will  never  instantiate  an  existential  variable  with  a  lambda-abstraction. 
Hence,  the  modal  substitution  6  has  the  following  form: 

6  (R\/u\,  •  •  •  ,  Rn/^n) 

where  R  denotes  a  neutral  atomic  object  which  can  be  a  bound  variable,  a  constant, 
an  application,  or  another  existential  variable  which  has  atomic  type.  Moreover,  since 
we  require  that  U\  (resp.  R\)  and  U-2  (resp.  R2)  are  in  canonical  form,  and  all  existential 
variables  are  of  atomic  type,  canonical  forms  are  preserved  when  applying  the  modal 
substitution  6  and  in  fact  we  can  check  if  {9]U\  is  dehnitionally  equal  to  \O\U2  by 
checking  syntactic  equality  between  these  two  objects.  Similar  statments  hold  for 
types,  kinds,  and  the  bound  variable  context  Ik 

Theorem  45 

1.  If  U  is  a  normal  term  and  all  modal  variables  are  of  atomic  type  and  A'  b  9  :  A 
and  A;  T  b  U  :  A  then  \6\U  is  a  normal  term. 

2.  If  R  is  a  neutral  term  and  all  modal  variables  are  of  atomic  type  and  A'  b  6  :  A 
and  A;  T  b  R  :  A  then  [0]/?  is  neutral. 

Proof:  Simultaneous  structural  induction  on  the  structure  of  normal  and  neutral 
terms.  The  key  observation  is  that  6  only  substitutes  neutral  objects  R  for  modal 
variables.  □ 

Theorem  46  Let  U  and  U'  be  normal  terms  and  R  and  R'  be  neutral  terms  and  all 
modal  variables  are  of  atomic  type  and  A  b  6  :  A'. 

1.  \e\u  =  1  «\u'  A;  [0]r  h  \e\u  =  mu'  ■.  \e]A. 

2.  [0]if  =  [O'] H  iff  A;  [S]r  h  \e\R  =  [0]i?  :  [0]A 

Proof:  By  the  previous  theorem,  we  know  \9\U  and  \9\U'  are  canonical.  As  dis¬ 
cussed  in  Chapter  2  on  page  61,  dehnitionally  equality  reduces  to  syntactic  equality 
for  canonical  forms.  □ 
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Higher-order  pattern  unification  can  be  done  in  two  phases  (see  [48,  22]  for  another 
account).  During  the  first  phase,  we  decompose  the  terms  until  one  of  the  two  terms 
we  unify  is  an  existential  variable  u[a\.  This  decomposition  phase  is  straightforward 
and  resembles  first-order  unification  closely. 


A -,T,x:A  b  Ui  =  U2  /  (A',  9) 

A;  r  P  Xx-.A.Ux  =  A x:A.U2  /  (A',  9) 


lam 


A;  T  lb  Rl  =  R2  /  (A',  9) 
A;  T  b  R1  =  R2  /  (A',  9) 


coerce 


A;  T  lb  x  =  x  /  (A,  idA)  VQr  A;  T  lb  c  =  c  /  (A,  idA)  COnSt 

A;  r  lb  R,  =  R2  /  (A1;  e1)  Ai;  [gi]T  b  Igi jjh  =  |gi]P2  /  (A2,  fl2) 

A;  r  lb  i?!  [/,  =  R2U2/  (A2,  [0!]  (02)) 

Note  that  we  do  not  need  to  worry  about  capture  in  the  rule  lam ,  since  existential 
variables  and  bound  variables  are  defined  in  different  contexts.  During  the  second 
phase,  we  need  to  find  an  actual  instantiation  for  the  existential  variable  u.  There  are 
two  main  cases  to  distinguish:  (1)  when  we  unify  two  existential  variables,  u[a\  =  v[a'] , 
and  (2)  when  we  unify  an  existential  variable  with  another  kind  of  term,  u[a]  =  U . 
In  the  following,  we  consider  each  of  these  two  cases  separately. 


3.1.1  Unifying  an  existential  variable  with  another  term 

The  case  for  unifying  an  existential  variable  u[c t]  with  another  term  U  can  be  trans¬ 
formed  into  u  =  [a]-1  U  assuming  u  does  not  occur  in  U  and  all  variables  v[t\  are 
pruned  so  that  the  free  variables  in  r  all  occur  in  the  image  of  cr  (see  below  or  [34,  22] 
for  details).  Before  considering  the  pruning  operation,  we  define  the  inverse  substitu¬ 
tion.  Note  that  we  view  [a]^1  U  as  a  new  meta-level  operation  such  as  substitution, 
because  it  may  be  defined  even  if  a  is  not  invertible  in  full  generality.  This  meta-level 
operation  is  defined  as  follows1. 

1We  do  not  omit  the  type  label  on  the  lambda-abstraction  here,  since  it  is  sometimes  convenient 
to  have  access  to  the  type  of  the  bound  variable  x.  This  is  not  strictly  necessary,  since  we  require  all 
terms  are  canonical  and  type  labels  can  be  omitted. 
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[a]  1  c  =  c 

[a\~X  x  =  y  if  x/y  E  cr,  undefined  otherwise 

[(t]_1(u[t])  =  u[[u]_1t] 

Wr'iRU)  =  ([a]-1  RUW}-1  U) 

[cr]  {\x\A.U)  =  Arc:([u]  A).[a,x/x\~  U  if  x  not  declared  or  free  in  u 

(•) 

(wr1^  [v]~lu/x) 

Note  that  the  defined  meta-level  operation  of  inverting  substitutions  is  only  defined 
for  a  variable  x  if  x/y  is  in  u  and  undefined  otherwise,  thus  enforcing  that  the  resulting 
term  [cr] —  1  C/  is  total  and  exists.  It  is  important  to  note  that  the  meta-level  operation  of 
inverting  substitutions  and  applying  the  modal  substitution  commute.  This  property 
only  holds  if  the  inverse  substitution  in  fact  exists. 

Lemma  47  (Property  of  inverting  substitutions)  Let  r  and  cr  be  pattern  substi¬ 
tutions. 

1.  If  [cr] — 1  r  and  [up1  (|0]t)  exist  then  [up1  (|0](t))  =  [0]([u]_1t). 

2.  If  [up1  U  and  [up1  ([#][/)  exist  then  [up1  ([#][/)  =  [0]([u]_1  U). 

3.  If  [u]  1  R  arid  [u]  1  (J0J.R)  exist  then  [a]  1  (J0J.R)  =  [0]([u]  1  R ). 

Proof:  By  simultaneous  induction  on  the  structure  of  r,  U .  and  R.  □ 

Lemma  48 

1.  If  [up1  r  exists  then  [u]([u]_1  r)  =  r. 

2.  If  [a]  1  U  exists  then  [u]([u]  1  U)  =  U . 

3.  If  [up1  R  exists  then  [u]([u]_1  R)  =  R. 

Proof:  The  lemmas  are  proven  by  simultaneous  structural  induction  on  r,  U,  and  R. 

□ 


u]  1(t,U/x)  = 
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Lemma  49 

1.  If  a  is  a  pattern  substitution  and  [a]U  =  U'  then  U  =  [a]  1  U' 

2.  If  a  is  a  pattern  substitution  and  [cr]i?  =  B!  then  R  =  [o']-1  R' 

Proof:  The  statement  is  proven  by  a  simultaneous  induction  on  the  structure  of  U 
and  R.  □ 

Lemma  50  Let  r  and  a  be  pattern  substitutions. 

1.  If  [o-]-1  r  exists  and  A;T  h  r  :  and  A;  T  b  a  :  T2,  then  A;  T2  b  [o']-1  r  :  i . 

2.  If  [o']  1  U  exists  and  A;  T  b  U  :  [a]Q  and  A;  T  b  a  :  T2,  then  A;  T2  b  [o']  1  U  :  Q. 

3.  If  [o']  1  R  exists  and  A;  T  b  R  :  [a]Q  and  A;  T  b  a  :  T 2 ,  then  A;  T2  b  [o']  1  U  :  Q. 

Proof:  By  simultaneous  structural  induction  on  the  definition  of  inverse  substitutions 
for  r,  U,  R.  □ 

Next,  we  discuss  pruning  for  normal  and  neutral  objects. 

A;  T  b  U  |  [o']-1  (A',  p )  Prune  U  with  respect  to  a 

A;  T  lb  R  |  [a]-1  =>•  (A',  p)  Prune  R  with  respect  to  a 

Let  A  be  the  modal  variables  in  U  (and  R  resp).  Then  pruning  with  respect  to 
the  substitution  a,  will  return  a  modal  substitution  p  s.t.  A'  b  p  :  A  and  wrHipW) 
exists.  The  modal  substitution  p  replaces  modal  variables  Ui  in  A,  where  uf[T }  occurs 
as  a  subterm  in  U  (or  R  resp.)  and  [o']-1  (ti,:[r])  n°t  defined,  with  new  modal 
variables  v[t'],  s.t.  t'  prunes  the  substitution  r  and  [o-]-1  (^[[tJt'])  exists.  In  other 
words,  pruning  ensures  that  all  bound  variables  occurring  in  \p\U  occur  in  the  range 
of  o'  which  is  achieved  by  applying  the  modal  pruning  substitution  p  to  U. 

A]T,x:A\-U\  [a,  x/x]~l  (Ai,  p\)  A;  T  lb  R  \  [o-]-1  (A',  p) 

A;  T  b  Xx:A.U  \  [a]"1  =»  (A1;  Pl)  A;  T  b  R  \  [o']-1  =4>  (A',  p) 

(Ai,n::TihQ,  A2);T  b  r  :  Ti  |  [a]-1  =»  T2  A’2  =  [idAl,  v'[\dy2]/v\  A2 
(Ai,n::TihQ,  A2);T  lb  u[r]  |  [o']-1  =>  ((A1?  r/::T2hQ,  A'2),  (idAl,  v'[\dy2]/v,  idAa)) 
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A;rihfl|  PE1  =>  (At,  Pi)  Ai;  jgjT  h  [gJC/i  |  [ct]"1  ^  (A2,  p2) 
A;rihfl(7|  [tr]-1  =s-(A2,  IpJ(ft)) 

[up1  x  exists 

A;  Yi,x\A,  r2  Ih  x  |  [cr]  1  =>•  (A,  idA)  A;  T  lb  c  |  [cr]  1  =>■  (A,  idA) 
Applying  a  modal  substitution  6  to  a  modal  context  A  is  defined  as  follows: 


[*](•)  =  ' 

p](w::Tm4,  A)  =  (u::[0]'I'h[0]A,  [0,u[id#]/u]A) 

if  u  does  not  occur  in  [0]  T  and  \9\A 

The  application  of  6  to  some  modal  context  A  is  only  defined  if  we  do  not  create 
any  circularities  in  the  resulting  context  [0]A.  This  ensures  that  the  resulting  modal 
context  [0]A  is  well-formed.  In  the  previous  rule  for  pruning  a  modal  variable  r;[r]  with 
respect  to  [up1 ,  we  create  a  new  context  A'2  =  [idA1,u/[id^2]/u]A2.  Since  the  range 
of  [idAj,  u/[id^r2]/u]  is  different  from  the  modal  variables  declared  in  A2,  we  always  will 
obtain  a  well-formed  A'2. 

Note  that  pruning  of  an  object  U  (resp.  R)  does  not  always  succeed.  In  particular, 
if  U  contains  a  bound  variable  x  which  does  not  occur  in  the  image  of  the  substitution  cr 
pruning  will  fail.  When  we  prune  an  object  U  (resp.  R),  we  assume  that  A;  T  b  U  :  A 
and  if  pruning  succeeds  it  will  return  a  new  context  A'  and  a  modal  substitution  p 
such  that  A'  b  p  :  A. 

Note  that  r  is  a  pattern  substitution  and  therefore  we  do  not  need  to  recursively 
prune  the  substitutions.  However,  we  may  need  to  prune  some  bound  variables  because 
[cr ]  1  t  may  not  necessarily  exist.  Next,  we  describe  the  pruning  of  a  substitution  r 
with  respect  to  a  substitution  [up1 . 

Given  a  substitution  r  such  that  A;T  h  r  :  Ti,  pruning  r  with  respect  to  [up1 , 
where  A;T  b  u  :  T  returns  a  context  T2  such  that  [up1  (p]id^2)  exists.  The  following 
judgment  describes  the  pruning  of  a  substitution  r. 

A;T  hr  :  V1  \  [up1  =y  T2 
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^2  is  well-formed  context  such  that  dr  2  <  dR  and  A;  dR  b  id^2  :  dR.  Pruning 
the  substitution  r  with  respect  to  [cr]-1  will  always  succeed  and  return  a  well-formed 
context  dR. 


A;  T  b  r  :  dR  |  [cr]  1  dR  [cr]  1  (y)  exists 
A;  T  b  •  :  •  |  [cr ]  1  =>  •  A;  T  b  (r,  y/x)  :  (dR,  x:A)  j  [a]-1  =►  (dR,  x:A) 

A;  T  b  r  :  dR  |  [cr]-1  =>•  1312  [cr]-1  (y)  does  not  exists 

A;  T  h  (t,  y/x)  :  (dR,  x:A)  \  [a]-1  =>  dR 

Before  we  show  some  properties  about  pruning,  we  summarize  some  of  the  consid¬ 
erations  in  defining  pruning  judgments.  Recall  that  we  call  pruning  when  unifying  an 
object  u[a]  with  another  object  U .  Since  we  only  unify  terms  of  the  same  type,  we 
know  that  A;  Y  b  u[a\  :  Q'  and  A;  T  b  U  :  Q' .  We  also  know  by  typing  invariant 
that  u::'Y\-Q  in  A  and  in  fact  A;T  b  u[cr\  :  [cr]Q  and  A;  T  b  cr  :  du  By  the  previous 
two  assumptions,  we  know  that  in  fact  Q'  =  [ a]Q .  Moreover,  A  =  Ai,  ur.^hQ,  A2  and 
Ap  d/  b  Q  :  type.  Since  cr  is  a  pattern  substitution,  we  must  have  Ap  Y  b  [a]Q  :  type. 

We  first  argue  why  we  do  not  need  to  prune  the  types  when  pruning  an  existential 
variable  v[r]  with  respect  to  [cr]  1  and  when  pruning  a  substitution  r  with  respect 
to  [cr]-1 .  Moreover,  we  show  when  pruning  r  with  respect  to  [cr]-1  the  resulting 
context  dR  must  be  well-formed.  We  say  a  variable  y  is  shared  between  two  pattern 
substitutions  r  and  a  if  y  is  in  the  image  of  both  r  and  cr.  It  is  easy  to  see  that  if 
A;  T  b  r  :  dR  |  [cr]  1  =>■  R/2,  then  dR  contains  exactly  those  variables  x  in  dR  such  that 
[t]x  is  shared  between  r  and  cr. 

Lemma  51  (Well-formedness  of  pruned  context) 

If  A;  T  b  t  :  dR  |  [cr]-1  =>•  dR,  A;  T  b  r  :  dR  and  A;  T  b  cr  :  dr  then  Ab$2  ctx  arid 
dr 2  <  dR. 

Proof:  Proof  by  structural  induction  on  the  first  derivation. 

Case  V  = - 

A;  r  b  •  :  •  |  [cr]  1  =>-  • 

A  b  •  ctx  by  rule 

•  <  • 
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A;  T  h  r  :  $!  |  [c]  1  dr2  [er]  1  (y)  exists 

A;Th  (r,y/x)  :  |  [er]-1  =>  {^!2,x\A) 


A;Th  (r,y/x)  :  (^i,x:^4) 

A;T  h  r  :  $!  and  A;  T  b  x:\t\A  (*) 
A;rhff:$ 

Ah$2  ctx  and  dr  2  <  d/i 

r  (y)  =  B=  [r]A 

er  =  (i ai,y/z,<72 )  and  dr  =  (d/',  z\A' ,  d/") 

r(s/)  =  BiaM' 

[t]A  =  [erx  ]A' 

[t]A  can  only  depend  on  variables  shared 

A  can  only  depend  on  variables  from  dr2 
A;  d^!  b  A  :  type 
A  b  dr2,  x:A  ctx 
(^2 ,x:A)  <  {^i,x:A) 


by  assumption 
by  inversion 
by  assumption 
by  i.h. 

by  inversion  on  (*) 
since  [er]-1  (y)  exists 
since  A;  T  b  er  :  dr 
transitivity  of  equality 

between  r  and  eri 

t  and  eri  are  pattern  substitutions 
see  remark  before  this  proof 
from  above 
by  rule 
by  i.h. 


Case  V 


A;  T  b  r  :  d>!  |  [er]  1  =>•  dr2  [er]  1  (y)  does  not  exist 

A;fb  (r,y/x)  :  ('h1,a::A)  |  [ex]-1  => 


A;Tb  (r,y/x)  : 

A;  T  b  r  :  dq  and  A;  V  b  y:[r]A 
A;rb(j:f 

A  b  dr 2  ctx  and  dr2  <  (dr^axA) 


by  assumption 
by  inversion 
by  assumption 
by  i.h. 

□ 


Lemma  52  (Well-formed  pruning  substitution) 

1.  If  A;  r  b  U  I  [er]-1  =>  (A',  p)  and  A;  T  b  U  :  [a]  A  arid  A;  T  b  er  :  dr 
then  A'  b  p  :  A. 

2.  If  A;  T  lb  R  |  [er]-1  =*  (A',  p)  and  A;  T  b  R  :  [a]  A  and  A;  T  b  er  :  d> 
then  A'  b  p  :  A. 
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Proof:  Simultaneous  structural  induction  on  the  first  derivation.  We  show  a  few  cases 
of  the  proof. 


A-,T,x:A\-U\  [cr,  x/x]  1  (A r,p) 

A;  T  b  \x:A.U  |  [a]-1  =>  (A',  p) 


A;  T  b  A x:A.U  :  [cr]B 

B  =  dtr:  B  | .  B  >  for  some  B  \  and  B2 

A;  T  b  [a]B  =  Ux:[a]Bi.[cr,  x/x]B2  :  type  and 

A;  T,  x:A  b  IJ  :  [cr,  x/x]B2 

A;  T  b  [a]B  =  Ilx:[a]Bi.[a}  x /x]B2  :  type  and 

A;  T  b  [cr]Bi  =  A  :  type  and  A;  T  b  [a,  x/x]B2 

A;T  b  a  :  ^ 

A;  T,  x:A  bd  :  $ 

A;  T,  x:A  b  x  :  A 

A;  T,  x:A  b  x  :  [<j]Bi 

A;  T,  x:A  b  (<r,  x/x )  :  (d>,  x:Bi) 

A'  b  p  :  A 


by  assumption 


by  typing  inversion  lemma  12 
by  equality  inversion  14 
[a,x/x\B2  :  type 

by  assumption 
by  weakening 
by  rule 
by  rule 
by  rule 
by  i.h. 


Case  V 


(A^uud/ihQ,  A2);T  b  t  :  |  [<r]  1  d>2 

(Ai,  A2);  T  lb  v[t]  |  [tr]-1  =►  (A ' ,  p) 


A'  =  (Ai,u/::d/2i-Q,  A'2)  and  A'2  =  [idAl,  u'[id*2]/u]  A2  and  p  =  (idAl,  u'[id*2]/u,  idAa( 


I'kihQ,  A2);T  b  v[t]  :  A 
id'll  Qi  A2);  T  b  r  :  dq  and 
idqi-Q,  A2);  r  b  [a]A  =  [t}Q  :  type 
:*ii-Q,A2);rb<7:tt 
A  b  d'o  ctx  and  d/2  <  d'l 
Ax  b  dq  ctx 
Ax  b  d'2  ctx 


(Ai,u: 

(Ai,  u: 
(Ai,  u: 

(Ai,  v: 


by  assumption 


A  —  fcr]  1  ([r]Q)  and  [a]  1  ([r]Q)  exists 


i-i 


by  typing  inversion  12 
by  assumption 
by  lemma  51 

by  well-typedness  of  (Ai,  irndqi-Q,  A2) 

since  d/2  <  d>i 
by  lemma  49 
should  not  affect  any  variables  in  Q.  Therefore, 


so  pruning  r  with  respect  to  [a 
Ai;  d/2  b  Q  :  type.  Moreover, 

(Ai,t/::d'2hQ,  A'2)  b  (idAl,7/[id*2]/u,  idAa)  :  (Ai,  uud'ihQ,  A2 
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Case  V 


A;  T  lb  R  |  [up1  =»  (A1;  pQ  Ap  [pi]£  b  \p.jU,  j  [a]"1  =»  (A2,  p2) 
A;  T  lh  RU  \  [a]"1  =►  (A2,  [p2](Pl)) 


A;  T  b  R  U  :  [a\A 
A;  T  h  i?  :  [u]  (ILciAi.^P)  and  A;T 
A;  T  h  [a,U/x\A2  =  [u]A  :  type 
Ai  b  pi  :  A 

Ap  [pi]r  b  \PljU  :  [pi]([a]Ai) 

Ap  [pi]r  b  [pi jU  :  [a]([pi]Ai) 

A2  b  p2  :  Ax 
A2  b  \p2\p\  :  A 


b  U  :  [cr]  Ai  and 


by  assumption 


by  typing  inversion  12 
by  i.h. 

by  substitution  property 
since  p\  is  a  pattern  substitution 

by  i.h. 

by  composition  of  modal  substitutions 


□ 


Moreover,  we  show  that  if  the  modal  variable  u  does  not  occur  in  R,  then  pruning 
R  with  respect  to  some  u,  results  in  the  modal  context  A'  and  the  pruning  substitution 
p,  where  u  will  be  mapped  to  itself. 

Lemma  53 

1.  If  the  m.odal  variable  u  does  not  occur  in  U  and  A;  T  b  U  |  [up1  =>■  (A*,p)  and 

A  =  (Ai,  u::^\-Q,  A2)7  then  A*  =  (A*,  u::[p]^/h[p](5,  AJ;)  and  p  =  (pi,  u[id[pj^]/u,  pi). 

2.  If  the  modal  variable  u  does  not  occur  in  R  and  A;  T  lb  R  |  [up1  =>■  (A*,p)  and 

A  =  Ai,  A2?  then  A*  =  (A*,  u::[p]'f'h[p]P,  Ap  and  p  =  (pi,  u[\dMq,]/u,  pi). 

Proof:  Simultaneous  structural  induction  on  the  pruning  derivation.  □ 

Next,  we  show  correctness  of  pruning. 

Lemma  54  (Pruning  —  Soundness) 

1.  If  A;  T  b  t  :  H/i  |  [up1  =>•  4/2  and  A;  T  b  r  :  d'l  and  A;  T  b  u  :  then 

[up  ([r](icp2))  exists. 

2.  If  A;  T  b  U  |  [up1  =>•  (A7,  p)  and  A;  T  b  U:A  and  A;  T  b  u  :  ^  then  [up1  (|p]C/) 
exists. 
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3.  If  A;  T  lb  R  |  [a]  1  ^  (A',  p )  and  A;  T  b  R:A  and  A;  T  h  a  :  f  then  [er]  1  (|p]i?) 
exists. 

Proof:  The  lemmas  are  proven  by  simultaneous  structural  induction  on  the  first 
derivation. 


Case  A;  T  lb  R  U  \  a'1  =>  (A2,  \p2jpi) 

A;  Tib  R  |  a-1  =>  (A^pi) 

[pilr  h  IpiW  \  ^_1  =>  (a2,p2) 

A;T  b  RU  :  A 

A;  T  b  R  :  Ylx:Ai.A2 

A;  r  b  U  :  A1  and 

A;  T  b  [idr,  U/x]A2  =  A  :  type 

Ai  h  pi  :  A 

Ai;  Mr  h  [Pl}U  :  [ftp, 

A 2  b  p2  :  Ai 
M-1  (IpijR)  exists 

(IpzKipiW))  exists 
M_1  (MPil^)  exists 
[P2lM_1  ([Pil-R)  exists 
MIpiP)  exists 
([[P2]piJ-R)  exists 

Case  A;  T  b  v[r]  \  a~1  =>•  (A',  p) 

A  =  A1,v::'SP-Q,  A2 

A'  =  (Ax,  v'::^2\-Q,  A'2)  where  and  A'2  = 

P  =  (idAl,7/[id*2]/u,idA2) 

A;  T  h  r  j  [a]”1 
A;  T  b  v[r]  :  A 
A;  T  h  r  :  T  and 
A;  T  b  A  =  [t]Q  :  type 
[n]-  ([r](id^2)  exists 


by  premise 
by  premise 
by  assumption 

by  typing  inversion  lemma  12 
by  lemma  52 
by  substitution  property 
by  lemma  52 
by  i.h. 
by  i.h. 

by  substitution  property 
by  substitution  property 
by  previous  lines 
by  substitution  definition 


[idAl,i/[id*2]/u]A2 

by  premise 
by  premise 
by  assumption 
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i/[[cr]  1  ([r]( id vp2))]  exists 
[o']-1  t/[[r](id^2)]  exists 
M"1  (M«M)  exists 

□ 


by  inverse  substitution  definition 
by  substitution  definition 


Lemma  55  (Pruning  —  Completeness) 

1.  If  A;  T  b  t  :  T  and  A;  T  h  tr  :  then  A;T  hr  :  $  |  [dp1  =>•  T2  for  some  T2. 

2.  If  A;  T  b  U  :  [<j\A  and  A;  T  b  a  :  T  and  A'  b  9  :  A  and  [o']-1  ([#][/)  exists,  then 
A;T  b  U  |  [o']-1  =>■  (A " ,  p)  for  some  A"  and  p  and  there  exists  a  substitution  p' 
such  that  6  =  \p'\p  and  A'  h  /  :  A". 

3.  If  A;  T  b  R  :  [o'JA  and  A;  T  b  o'  :  T  and  A'  b  9  :  A  and  [o']-1  exists,  then 

A;T  b  R  |  [o']  (A",p)  for  some  A"  and  p  and  there  exists  a  substitution  p' 

such  that  0  =  \p'\p  and  A'  b  p'  :  A". 

Proof:  The  statement  is  proven  by  simultaneous  structural  induction  on  the  typing 
derivation.  We  show  a  few  cases. 

Case  A;  T  b  v[t]  :  [<j\A 

A;T  h  r  :  $!  by  typing  inversion  lemma  12 

A  =  (A1;  A2)  and  A;T  h  [o]A  =  [r\Qi  :  type 

A;  T  b  o'  :  T  by  assumption 

A;  r  b  r  :  Ti  I  [o']-1  T2  by  (1) 

A;  T  b  v[t]  I  [o']-1  =>  ((A1,v,::'&2'-Qi,  A'2),  (idAl ,u'[id<p2]/u,  idA/J)  by  rule 

Ai;  T2  b  Qi  :  type  since  [<j\A  =  [t]Q\  and 

T2  contains  exactly  those  variables  shared  between  a  and  r 
A'  b  (6*i,  U/v,  62)  :  (Ai,  vv.'hp-Qi,  A2)  by  assumption 

let  p'  =  (9i,  U/v',  d2)  then  we  have  6  =  [p']((idAl,  t/[id^2]/u,  idA/J. 

Case  A;  T  b  Xx:A.U  :  [ a}B 
A;  T,  x:A  b  U  :  [a,  x/x)B2  and 

A;  T  b  [cr]B  =  Ylx:A.[a,x/x\B2  :  type  by  typing  inversion  12 
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A;  T  b  [a\B  =  Ux:[a\Bi.[a,x/x\B-2 
A;  T  b  A  =  [ cr]B\  :  type 

A;rhff:f 

A;  T,  x:A  b  x  :  A 

A;  T,  x:A  b  x  :  [(?\Bi 

A;  T,  x:A  b  (cr,  x/x)  :  (T,  x:Bi) 


:  type  and 


a]  1  ([0](Ax:A{7))  exists 


Ax: [a]  1  ([0]t4).([ct, x/x]  1  [$]£/)  exists 


-l 


( [cr,  x/x]  1  \0\U)  exists 
3p' .9  =  [p']p  and  A;T,x:A  b  U  \  [cr,  x/x]-1 
A;T  b  A X-.A.U  I  [cr]-1  =>  (A', p) 


by  equality  inversion  14 
by  assumption 
by  rule 
by  rule 
by  rule 
by  assumption 
by  definition  of  inverse  substitution 


(A  r,p) 


by  i.h. 
by  rule 


□ 


Now,  we  can  give  the  cases  for  unifying  an  existential  variable  with  another  kind  of 
term.  Since  we  are  requiring  that  all  existential  variables  are  of  atomic  type,  this  term 
R  can  only  be  a  modal  variable  v[t'\,  where  v  is  different  from  u,  an  application,  an 
ordinary  variable  or  a  constant. 


A;T  b  R 


c 


i-i 


(A-.  P ) 


u  does  not  occur  in  R 


A*  =  (A^::[p]tth[p]Q,A£) 
A'2  =  [id a*,  [a]-1  (lp}R)/u}A*2 


exl 


(A1,'u::'khQ,  A2);T  lb  w[a]  =  R  /  ((A*,  A'2),  [idA*,  [a]  1  ({pjR)/u,  idA*]p) 


A;T  b  R  |  [a]’1  =►  (A*,  p)  A*  =  (AJ, i/::[p]4>h[p]Q,  AJ) 

u  does  not  occur  in  R  A2  =  [idA*,  [cr]-1  ([p]7?)/n]A2 

- 6x2 

(Aijuutfi-Q,  A2);T  lb  R  =  u[a]  /  ((A*,  A2),  [idA*,  [cr]-1  ([p]R)/-u,  idA*]p) 

The  side  condition  “u  does  not  occur  in  R”  ensures  that  we  obey  the  occurs- 
check.  Note,  that  we  do  not  need  to  ensure  that  u  does  not  occur  in  the  type  of 
R.  Recall  that  by  invariant,  we  know  that  A;  T  b  u[a\  :  Q'  and  A;  T  b  R  :  Q' . 
We  also  know  by  typing  invariant  that  u::'$/\-Q  in  A  and  A;  T  b  xt [cr]  :  [a\Q  and 
A;  T  b  cr  :  T.  By  the  previous  two  assumptions,  we  know  that  Q'  =  [ cr\Q .  Moreover, 
A  =  Ai,  u::^hQ,  A2  and  Ay  T  b  Q  :  type.  Since  cr  is  a  pattern  substitution,  we  must 
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have  Ai;T  b  [a]Q  :  type.  Since  [a]Q  =  Q',  we  know  that  u  cannot  occur  in  the  type 
Q! .  Therefore  we  do  not  need  to  perform  the  occurs  check  on  the  type  of  R. 

However,  since  there  may  be  dependencies  among  the  types  and  declarations  in  A2 
may  depend  on  u,  it  is  not  necessarily  obvious  why  traversing  R  and  checking  whether  u 
occurs  in  R  suffices  in  the  higher-order  dependently  typed  setting.  Potentially  we  could 
be  in  the  following  situation:  a  modal  variable  v[t'\  occurs  in  R,  but  vw^RQ'  occurs  in 
A2  and  depends  on  u.  When  we  construct  the  new  A2  =  [id ,  [cr]  1  ([p]i?)/ti]A2,  we 
will  replace  occurrences  of  u  in  T7  and  Q'  with  [cr]_1  (|p]i?).  But  since  R  itself  refers  to 
v,  we  may  be  creating  a  circular  new  declaration  for  v  and  the  resulting  modal  context 
A2  may  not  be  well-formed. 

Lemma  56 

1.  If  A;T  b  U  4=  A  and  A  =  (A1;  A2)  and  u  does  not  occur  in  T,  U,  and 

A,  then  for  any  modal  variable  v  where  v  is  different  from  u  that  occurs  in  U  and 
vr.tyRQ'  is  declared  in  A,  u  does  not  occur  in  T7  and  Q' . 

2.  If  A;  T  b  R  =>  A  and  A  =  (Ai,  A2)  and  u  does  not  occur  in  T,  R,  then 

u  does  not  occur  in  A  and  for  any  modal  variable  v  where  v  is  different  from  u 
that  occurs  in  R  and  w.-.^RQ'  is  declared  in  A,  u  does  not  occur  in  T7  and  Q' . 

Proof:  By  simultaneous  induction  on  the  algorithmic  typing  derivation. 

Case  A;  T  b  R  U  =>  [idr,  U/x\A2 
A;  r  b  R  =*►  Ux:A[.A2 

A;  T  b  U  Ai  and  A;  T  b  Ai  A\  :  type  by  inversion 

A\  =  A\  since  Ai,  A\  are  canonical 

u  does  not  occur  in  R  U  by  assumption 

u  does  not  occur  in  R 
u  does  not  occur  in  U 

u  does  not  occur  in  G  by  assumption 

u  does  not  occur  in  Ax'.A\  ,A2  and  for  any  modal  variable  vr.fyRQ'  occurring  in  R, 
u  does  not  occur  in  T7  and  Q'  by  i.h. 

u  does  not  occur  in  A\ 
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u  does  not  occur  in  A\  since  A\  <==>  A\ 

for  any  modal  variable  v::'T'\-Q'  occurring  in  U, 

u  does  not  occur  in  T7  and  Q'  by  i.h. 

for  any  modal  variable  v::xT'\-Q'  occurring  in  R  U, 
u  does  not  occur  in  T7  and  Q' 


Case  A");  T  b  v[r]  =>-  [r]Q' 

A  =  (A',  v::'B'hQ',  A")  and  A;rhr^f' 
u  does  not  occur  in  T 
u  does  not  occur  in  T7 
u  does  not  occur  in  [t]Q’ 
u  does  not  occur  in  Q' 


by  inversion 
by  assumption 
since  r  is  a  pattern  substitution 
by  assumption 
since  r  is  a  pattern  substitution 


Case  A;  V  b  c  A 

c  is  defined  in  the  signature  E.  E(c)  =  A.  Moreover,  A  must  be  closed  and  cannot 
refer  to  any  modal  variables.  Therefore,  u  cannot  occur  in  A. 

Case  A;Tbi^d 
r(x)  =  A 

u  does  not  occur  in  A 
Case  A;  T  b  A x:Ai.U  <£=  Tlx-.A1.A2 

A;  T,  x\A\  b  h  A2  by  inversion 

u  does  not  occur  in  T  by  assumption 

u  does  not  occur  in  Tlx-.A1.A2  by  assumption 

u  does  not  occur  in  A\  and  A2 
u  does  not  occur  in  T,  x:A± 
for  any  v::xT'hQ'  occuring  in  U 

u  does  not  occur  in  T7  and  Q'  by  i.h. 

for  any  vw^RQ'  occuring  in  \x\Ai.U 
u  does  not  occur  in  T7  and  Q' 


by  inversion 
since  u  does  not  occur  in  T 
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□ 

The  last  lemma  justifies  why  a  simple  occurs  check  also  suffices  in  the  higher-order 
dependently  typed  setting.  In  an  implementation,  we  may  combine  computing  the 
pruning  substitution  with  the  occurs  check.  Since  we  already  traverse  the  term  R  to 
check  whether  [cr]  1  R  exists,  we  may  simultaneously  check  whether  u  occurs  in  R. 
However,  the  previous  lemma  shows  that  for  higher-order  patterns,  we  do  not  need  to 
prune  types  and  the  bound  variable  context.  Moreover,  we  do  not  need  to  perform  an 
occurs  check  on  the  types  of  modal  variables  which  may  occur  in  R2  . 

3.1.2  Unifying  two  identical  existential  variables 

For  unifying  the  two  existential  variables  which  are  the  same,  we  need  an  additional 
auxiliary  judgment,  which  computes  the  intersection  of  two  substitutions. 

A;rbo-nr:'F=^'I'/ 

Given  the  substitution  A;  T  b  cr  :  T  and  A;  T  b  r  :  T,  we  construct  a  context  T7 
which  is  well-formed  and  for  the  substitution  id^/  such  that  A;  T  b  id^/  :  T7  we  have 
[cr](id#/)  =  [r](id#/). 


A;  T  b  •  fl  •  :  •  • 

A^brntr^^T 

A;  T  b  (r,  y/x)  D  (cr,  y/x)  :  (T,  x:A)  =>-  (T7,  x:A) 

A;rbrricr:T=^'I/7  z  ^  y 
A;  T  b  (r,  z/x)  fl  (cr,  y/x)  :  (T,  x:A)  =>•  T7 

We  say  a  variable  y  is  strictly  shared  between  two  pattern  substitutions  r  and  cr  if 
there  is  an  x  such  that  y  =  [r\x  =  [a]x.  It  is  easy  to  see  that  if  A;  T  b  r  fl  cr  :  T  T7 
then  T7  contains  exactly  those  variables  x  in  T  such  that  y  =  [t\x  =  [a\x  (in  other 
words,  y  is  strictly  shared  between  r  and  cr).  The  resulting  context  T7  will  always  be 
well-formed.  To  illustrate  the  difference  between  the  notion  of  ’’shared”  and  ’’strictly 

2It  seems  possible  that  an  extended  occurs  check  would  lead  to  incompleteness,  no? 
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shared”  consider  the  substitution  [y/xl,  z/x2]  and  [z/xl,y/x2].  Here,  y  and  z  are 
shared,  but  not  strictly  shared,  y  and  z  will  be  retained  under  inverse  substitution, 
but  they  are  pruned  under  intersection. 

Lemma  57  If  A;  T  b  r  fl  a  :  d'  d'7,  A;Thr  :  $  and  A;  Y  b  a  :  'h  then  Ahf'  ctx. 

Proof:  Structural  induction  on  the  first  derivation.  We  consider  the  interesting  case, 
where 


A;rhrna 

A;  T  h  (t,  y/x)  D  (cr,  y/x )  :  (d>,  x:B)  =>•  \P7,  x\B 


A;Th  (r,y/x)  :  (V,x:B) 

A;  T  h  r  :  d' 

A;Th  y.[r}B 
h(y)  =  B'  =  [t]B 
A;Th  (a, y/x)  :  (V,x:B) 

A;rh(T:f 
A;T\-y:[a]B 
r(y)  =  B'  =  [o)b 

B’  =  [t\B  =  [a]B 

B  can  only  depend  on  variables  strictly  shared  between  r  and  o 


by  assumption 
by  inversion 
by  inversion 
by  inversion 
by  assumption 
by  inversion 
by  inversion 
by  inversion 
by  previous  lines 


A  h  ctx  by  i.h. 

A;  h  fl  :  type  since  d'7  contains  exactly  those  variables  strictly  shared  by  r  and  o 
A  b  (d/',  x:B)  ctx  □ 


Next,  we  show  correctness  of  computing  the  intersection  between  two  substitutions. 


Lemma  58 

1.  If  A;  T  b  r  :  d'  and  A;  Y  b  a  :  'b  and  A  =  Ai,  w::d/hQ,  A2  then 

A;rbrncr:d'=^d'7 

2.  If  A;  T  b  r  fl  cr  :  d^  =>•  d/7  then 

[cr] (id^>/)  =  [r](id^)  and  A;  d'  b  id^/  :  d/7. 
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Proof:  The  first  statement  is  proven  by  structural  induction  on  the  first  derivation. 
The  second  statement  is  proven  by  induction  on  the  first  derivation  using  lemma  4.  □ 

Now  we  can  give  the  rule  for  unifying  two  modal  variables  which  are  the  same. 

(Ai,  ur.'ff\-Q,  A2);T  b  r  D  cr  :  T  =>•  T2  A2  =  [idAl,  u,[id^2]/n]A2 
(Ai,  u::^\-Q,  A2);  T  lb  u[a]  =  u[t]  /  ((Ai,  v::^2\-Q,  A2),  (idAl,  v[\dq,2]/u,  idA2))  6X3 

We  have  shown  that  the  resulting  'i>2  of  computing  the  intersection  of  r  and  a,  is 
indeed  well-formed  (lemma  57).  We  can  also  justify  why  in  the  unification  rule  itself 
the  type  Q  of  two  existential  variables  must  be  well- typed  in  the  pruned  context  T2. 
Recall  that  by  typing  invariant,  we  know  that  A;  T  b  r  :  T  and  A;T  b  u  :  $  and 
[a]Q  =  [t]Q  =  Q' .  But  this  mean  that  Q  can  only  depend  on  the  variables  strictly 
shared  by  r  and  cr.  Since  T2  is  exactly  the  context,  which  captures  the  shared  variables, 
Q  must  also  be  well-typed  in  T2. 

During  unification,  we  ensure  that  the  resulting  modal  substitution  9  maps  all  the 
modal  variables  in  A  to  a  new  modal  context  A'. 


Lemma  59 

1.  If  A;  T  b  Pi  =  U2  f  (A',  9)  and  A;  T  b  IR  :  A  and  A;  T  b  U2  :  A 
then  A'  b  9  :  A. 

2.  If  A;  T  lb  Ri  =  R2  /  (A',  9)  and  A;  T  b  Ux  :  A  and  A;  T  b  U2  :  A 
then  A'  b  6  :  A. 

Proof:  Structural  induction  on  the  first  derivation.  We  show  the  cases  for  unifying  an 
existential  variable  with  another  term  and  unifying  two  identical  existential  variables. 

Case  V  =  A;  T  b  u[a]  =  R  /  ((A*,  A'2),  [idA«,  [a]-1  ({pjR)/u,  idA*]p) 

A  =  (Ai,  A2) 

A;  T  b  R  |  [u]-1  =r>  (A *,p)  and  u  does  not  occur  in  R  by  premises 

A;  T  b  u[a\  :  A  by  assumption 

A;  T  b  a  :  T  and  A;  T  b  A  =  [a\Q  :  type  by  typing  inversion  lemma  12 

u  does  not  occur  in  T  and  Q  by  well-formedness  of  A 

A;  T  b  R  :  [a\Q  by  assumption 
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A*  b  p  :  A 


or 


1-1 


[p]i?  exists 


by  lemma  52 
by  lemma  54 

A*  =  (A],w::[p]'bh[p]]Q,  Ap  and  p  =  (p{,  u[idMq,\/u,  p*2)  by  lemma  53 

A*;  [p]T  b  [p]i?  :  [p]([cr]Q)  by  substitution  property  (lemma  41) 

A*;  [p]r  b  [p]i?  :  [cr]([p](3)  since  cr  is  a  pattern  substitution 

A*;  [p]T  b  cr  :  [pj 4/  by  substitution  property 

A*;  [pj'b  b  [cr]_1  (|p]i?)  :  \p\Q  by  lemma  50 

u  does  not  occur  in  [p]4/  and  [p]  Q  and  [cr]-1  (|p]i?  because  p  only  replaces  modal 
variables  with  new  variables 

for  any  modal  variable  v  occuring  in  [up1  (|p]-R)  where  v::4/5 -Q', 

u  cannot  occur  in  4/'  and  Q'  by  lemma  56 

i-i 


A2  =  [idA*,  [cr]  MR)/u}A*  and  b  A2  mctx 


by  previous  lines 
i.e.  no  circularities  are  created  in  A2 
6  =  [idA*,  [cr]  1  ([ p]R)/u,  id a*]p  =  (Pi,  [cr]  1  ([p]i?)/u, p2)  by  substitution  definition 


(A],  A2);  [p]4/  b  [cr]  1  (|p]i?)  :  \p\Q  since  u  does  not  occur  in  4/,  R  or  Q 

(Ai?  Ap;  [pO'b  b  [cr]-1  (IpjR)  :  \pljQ  since  Ajj^bQ:  type 

(A],  A2)  b  p\  :  Ai  since  the  pruning  substitution  p*  only  instantiates 

modal  variables  from  A  with  new  modal  variables 
(A],  A2)  b  (p],  [cr]-1  ( [p]R)/u )  :  (Ai,  ur.^hQ)  by  previous  lines 

(A],  A')  b  (p],  [cr]-1  ([  p]R)/u,  p*2)  :  A  by  typing  rules 


Case  V  =  A-,T  \r  u[a]  =  u[t]  /  ((Ai,  v::4/2i-(2,  A2),  [idAl,  u[id*2]/u,  idA2]) 

A  =  (Ax,  u::^hQ,  A2) 

A;  T  b  r  fl  cr  :  41  =4*  4^2  by  premise 

A;  T  b  u[r]  :  A  by  assumption 

A;T  b  r  :  $  and  A;  Y  b  A  =  [t]Q  :  type  by  typing  inversion  12 

A;  T  b  u[a]  :  A  by  assumption 

A;  T  b  cr  :  4r  and  A;  Y  b  A  =  [ a]Q  :  type  by  typing  inversion  lemma  12 

A;  T  b  [a]Q  =  [t\Q  :  type  by  transitivity 

Q  can  only  depend  on  those  bound  variables  which  are  strictly  shared  by  r  and  cr 
4b  exactly  contains  those  shared  variables 

A].;  4/  b  Q  :  type  by  well-typedness  of  A 
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Ai;  'I/2  I-  Q  '■  type  by  previous  lines 

(Ai,  vr.^^Q,  A'2)  b  (idAi,  u[id^2]/u,  idA2)  :  A  by  previous  lines  and  typing  rules 


□ 


Recall,  we  assume  that  the  two  objects  we  unify  are  in  canonical  form,  in  particular 
we  may  omit  the  type  label  on  A-abstractions. 

For  soundness,  we  show  that  if  two  objects  U\  and  U2  unify  under  the  modal  sub¬ 
stitution  9,  then  \9\U\  is  definitional  equal  to  [[6?]  C/2  -  Since  we  concentrate  on  canonical 
forms,  definitional  equality  can  be  established  by  checking  whether  two  objects  are 
syntactically  equal.  Now  we  show  that  if  the  object  U\  and  U-2  unify  under  the  modal 
substitution  9 ,  then  \9\U\  is  syntactically  equal  to  [0][/2,  thereby  showing  indirectly 
that  \6\U\  and  \9\U2  must  also  be  definitional  equal. 

Theorem  60  (Soundness  of  higher-order  pattern  unification) 

1.  If  A;  T  b  Ux  =  U2  /  (A',  9)  and  A;  T  b  Ux  :  A,  and  A;  T  b  U2  :  A 
then  {ejUi  =  [0\U2. 

2.  If  A;  rib  i?!  =  R2  /  (A',  9)  and  A;  T  b  Rx  :  A,  and  A;  T  b  R2  :  A 
then 


Proof:  By  simultaneous  structural  induction  on  the  first  derivation.  Below,  we  give  a 
few  cases. 


Case  V 


A;  T,  x:A  b  Ui  =  U2/{ A',  9) 

A;  T  b  A X-.A.IR  =  Xx:A.U2/ (A',  9) 


A;  r  b  \x\A.U\  :  A' 

A;  T,  x:A  b  Ui:B 

A;  T  b  A  :  type  and  A;  T  b  A'  =  Ux:A.B  :  type 
A;  T  b  A x:A.U2  :  A' 

A;  T,  x:A  \-  U2  :  B 

[#i]G  =  [«]  u2 


by  assumption 
by  inversion  lemma  12 

by  assumption 
by  inversion  lemma  12 
by  i.h. 
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A'  b  0  :  A 

MA  =  [0\A 

Xx:[0\A.[0\Ui  =  Xx:[0\A.mU2 

l9j{Xx:A.U1)  =  m(Xx:A.U2) 


by  lemma  52 
by  reflexivity 
by  syntactic  equality 
by  modal  substitution  definition 


Case  V 


A;  rib  R,  =  R2/(  A1,e1) 

A;  r  lb  ux 


ai;  [gijr  b  lQi]u^=  mu2/(A2, 92) 

=  R2U2/(A2 ,  [021(0!)) 


A;  r  b  R1  Ux  :  A 
A;  T  b  R2  U2  :  A 
A;  T  b  Ri  :  II x:A2.A1 

A;  T  b  Ui  :  A2  and  A;  V  b  [idr,  U\/x]Ai  =  A  :  type 


by  assumption 
by  assumption 
by  inversion  lemma  12 


A;  T  b  R2  :  Yix\A2.Ai 

A;  T  b  U2  :  A2  and  A;  V  b  [idr,  U2/x]Ai  =  A  :  type 

le.jR,  =  [0,]  r2 

A2  b  d2  :  Ai 

mmRi)  =  mmiRi)  by 

mm*)  =  meMR*) 

Ax  b  0!  :  A 

Ax;  [0xlr  b  [0x]C/x  :  [0i]A2  by 

Ax;  [0x]r  b  [0x]C/2  :  [0i]A2  by 

mmui)  =  [02]([0i  ju2) 

[[02]0i][/i  =  [[02]0i]c/2 

[[02]0x](i?x  f/x)  =  [[02]0i](i?2  f/2) 


by  inversion  lemma  12 

by  i.h. 
by  lemma  52 
modal  substitution  lemma  41 
by  modal  comp,  lemma  42 
by  lemma  52 
modal  substitution  lemma  41 
modal  substitution  lemma  41 

by  i.h. 

by  modal  comp,  lemma  42 
by  rule  and  modal  sub.  def. 


Case  V  =  (Ax,  A2);  V  lb  u[a]  =  R  /  (A',  0) 

A'  =  (A*,  A')  and  0  =  [idAp  [a]-1  ([  p\R)/u,  idA*]p  by  premise 

A;  T  b  R  |  [a]-1  ^  (A*,  p )  and  by  premise 

A*  =  (A*,  rt::[p]\l/i-[p](5,  A2)  and  u  does  not  occur  in  R  by  inversion 

A*  b  p  :  A  by  lemma  51 

p  =  (pi,  tt[id[p]*]/tt,  p2)  by  lemma  53 
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A'  b  9:  (A^u-tfi-Q.Aa) 
(M_1  (M-R))  exists 
Ai,  ur.^hQ,  A2;  T  b  u[a]  :  Q' 
A;  T  h  Q'  =  [a]Q  :  type 
Al,u::^Q,A2-r\~  R:Q' 
A';  [9}T  h  [0]i2  :  [0]Q' 

=  M(M_1  HR) 

=  MR 
=  MR 


by  lemma  59 
by  lemma  54 
by  assumption 
by  typing  inversion  lemma  12 
by  assumption 
by  modal  substitution  lemma  41 
by  substitution  definition 
by  lemma  48 
because  u  does  not  occur  in  R 


Case  D  =  (A1;  u::^\-Q,  A2);  T  lb  u[a\  =  u[r]  /  (A',  9) 
A’  =  (A1,v::'&2'~Q,  A'2)  and  9  =  (idAl,  u[id*2]/u,  idAa) 
(Ai,  A2);  n-rfla  :  4  $2  and  A'2  =  [idAl, 

Hid*!,  =  [w]  idvj>2 

v[[o-]id*2]  =  u[[T]id®a] 

H(u[id*a])  =  [r](u[id^2]) 

MOM)  =  [0](u[t]) 


by  premise 
u[id^2]/,u]  A2  by  premise 
by  lemma  58 
by  syntactic  equality 
by  substitution  definition 
by  substitution  definition 


□ 

For  completeness,  we  again  assume  that  the  objects  are  in  canonical  form  and 
show  that  if  two  objects  are  syntactically  equal  under  some  modal  substitution  9  then 
they  are  unifiable  and  the  unification  algorithm  returns  a  modal  substitution  9 1  s.t. 

9  =  W- 


Theorem  61  (Completeness  of  higher-order  pattern  unification) 

1.  If  A;  T  h  U\  :  A  and  A;  T  b  U2  :  A  and  A'  b  9  :  A  and  {6}Ui  =  \0\U2  then 

for  some  A"  and  9',  we  have  A;  V  b  U\  =  U2  /  (A",  O')  and  there  exists  a  modal 
substitution  p,  s.t.  A1  b  p  :  A”  and  9  =  [p](0'). 

2.  If  A;  T  b  Ri  :  A1  and  A;  T  b  R2  :  A2  and  A'  b  9  :  A  and  [0]l?i  =  [0]-R2  then 
for  some  A "  and  9',  we  have  A;  Y  lb  Ri  =  R2  /  (A",  O'),  [0]^!  =  \0\A2  arid 
there  exists  a  modal  substitution  p,  s.t.  A'  b  p  :  A"  and  9  =  [pJH). 
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Proof:  Proof  by  simultaneous  induction  on  the  structure  of  U\  and  U2  (and  R\ ,  R2 
respectively).  Most  cases  are  straightforward  and  we  give  a  few  cases  to  illustrate. 

Case  Ux  =  A x.U'  and  U2  =  A x.U" 


m(Xx.U')  =  [6>]  (A  x.U") 

\x.[9}(U’)  =  Xx.[  9\U" 

mm  =  mm) 

A;  r  I—  A  x.U1  :  A 
A;r,x:A  b  U':A2 

A;  T  b  A\  :  type  and  A;  Y  b  A  =  Ylx-.A1.A2  :  type 
A;  r  h  A  x.U"  :  A 
A;  T,  x:A[  b  IJ"  :  A'2 

A;  T  h  A[  :  type  and  A;  T  h  A  =  Ylx-.A\  ,A’2  :  type 


by  assumption 
by  substitution  definition 
by  definition  of  syntactic  equality 
by  assumption 
by  inversion  lemma  12 

by  assumption 
by  inversion  lemma  12 


A;  T  h  Ylx-.A1.A2  =  Ylx\A\.A’2  \  type  by  transitivity 

A;  T  b  Ai  =  A\  \  type  and  A;  T,  x\Ai  b  A2  =  A'2  :  type  by  injectivity  of  products 

(lemma  15) 

A;  T,  x:A]_  h  U'  =  U"  /  (A",  O’)  by  i.h. 

9  =  [p](0')  and  A"  b  9'  :  A  and  9  =  [p](0') 

A;  T  b  A  x.U’  =  A  x.U"  /  (A",  9’)  by  rule 


Case  Ci  =  /?,  U[  and  U2  =  li2  U’2 


A;  T  b  Ri  U[  -.  A  by  assumption 

A;  T  b  C  :  Y\x\Ai.A2  by  typing  inversion  lemma  12 

A;  T  b  U[  \  Ai  and  A;  Y  b  A  =  [idr,  U\/x\A2  :  type 

A;  T  b  R2  U2  :  A  by  assumption 

A;  T  b  R2  :  Ylx:A\  ,A2  by  typing  inversion  lemma  12 

A;  T  b  U2  :  A[  and  A;  Y  b  A  =  [idr,  U2/x\A'2  :  type 


A'  b  9  :  A  by  assumption 

[d](i?i  U[)  =  [d](i?2  U2 )  by  assumption 

(|0]i?i)  (|0]C{)  =  (|0]i?2)  (I^JC^)  by  substitution  definition 

(J^JCi)  =  (|0]i?2)  and  (|^]C()  =  mU'2)  by  definition  of  syntactic  equality 
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A;T  lb  R1  =  _R2/(Ai,  9\)  and  [P](Ihr:Ai.A2)  =  [0](II x:A'vA'2)  by  i.h. 

dp. A'  b  p  :  Ai  and  9  =  \p\6\ 


Ai  b  91  :  A 
Ai;  [fljr  b  [0JU1  : 

Ai;  [#jr  h  :  IftK, 

=  ifcpK.pp:, 

IMA  =  MA 

[[pl^il^i  =  [[p]#i]^i  and  [#i]^4i  = 

[mw  =  mw 

\pimui)  =  \pw  iM) 

ai;  pjr  b  [0i]l7{  =  mu'/i  a2,92) 

3p'.p  =  Ip'\62  and  A'  b  p'  :  A2 

A;  T  lb  (i?,  U[)  =  (R2  £/')/(A2,  [02]0O 

d  =  =  M(N^i) 


by  lemma  59 
by  substitution  property  (lemma  41) 
by  substitution  property  (lemma  41) 
by  substitution  property 
by  syntactic  equality 
by  previous  lines 
by  previous  lines 
by  substitution  definition 
by  i.h. 

by  rule 

by  substitution  definition 


Case  Ui  =  u[a]  and  u  does  not  occur  in  U2  By  assumption,  the  type  of  U2  must  be 
atomic,  that  is,  U2  =  R2. 


[#J(«M)  =  [9}R2 
e  =  (e1,R/u,e2) 

A  =  Ai,u::'&\-Q,  A2 
[a]R  =  {9]R2 
R  =  [a]"1 19]R2 

9  =  ( 91,R/u,92 )  =  (9U  [a]"1  19}R2/u,92) 

A;  T  b  u[a\  :  [<j\Q 

A;rbtr:f 

A'h(9,,[<T]-1[0]fl2/ii,02):A 

A';  I9]>P  h  [ct]^1  [0]fl2  :  [0]Q 
A';  [0]r  h  a  :  [»]* 

A';  [OF  h  Hla]-1  [9]JS2  :  [<r][0]Q 
A';  [9]r  h  \B\R2  :  \0\[a]Q 


by  assumption 
by  assumption 
by  assumption 
by  substitution  definition 
by  lemma  49 
by  previous  assumptions 
by  assumption 
by  typing  inversion 
by  assumption 
by  typing  rules 
by  weakening 
by  modal  substitution  property 
by  substitution  property 
by  lemma  44  and  48 
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A;  T  b  R2  :  [a]Q 
A;  T  b  i?2  |  [a] 


by  modal  substitution  property 


1-1 


(A*,  p)  and 


3 p'.  6  =  [p']p  and  A'  b  p'  :  A* 

A*  b  p  :  A 

p  =  (pi,u[idw^]/u,p2)  and  A*  =  (AJ,  u::[p]^h[p]Q,  A*) 


by  lemma  55 
by  lemma  52 
by  lemma  53 


A;  T  lb  u[a]  =  R2  /  ((AJ,  A'2),  [id a*,  [cr]  1  ([p]^)/^,  idA*]p) 


by  rule 
by  previous  lines 
by  previous  lines 
by  substitution  definition 
since  by  lemma  54  [cr]-1  ([p]i?2)  exists 


with  A'2  =  [id a*,  [cr]  1  (lp]R2)/u,  idA*]A; 

e  =  {e1,R/u,e2)  =  ^[a]-1  mR2)/u,e2) 

=  (^i,  [cr]_1  (llp'}p}R2)/u,02) 

=  (0i,  M-1  (Ip11pIr2)/^o2) 

=  (0i,  [p'lM-1  {Ip\R2)/u,o2) 

and  by  lemma  47,  [cr]-1  ([p']  [p] i?2)  =  [p'l[cx]-1  ([p]i?: 

=  [p> 

=  lpl(pi,u['dM<f,}/u,p2) 

=  ([p'Ipi,  Ip1(u[^M^})/u,  [p']p2) 

Since  (Pi,  [p'][cr]-1  ({p]R2)/u,  02)  =  ([p']pi,  |>/] (w[id [pj«r] )/«, 
we  must  have  the  following: 

[p'](w[id*])  =  [p']([cr]-1  ([p] R2))  and 
p'  =  (p'i,\p'm'1mR2))/u,p'2) 

=  [p'i,py(idAp  ([cr]-i  ({p\R2))/u,  idA*) 

and  the  fact  that  u  does  not  occur  in  R  itself 
0  =  [[pl,P2l(idAp  ([cr]  1  (lp]R2))/u,  idA*)]p  by  previous  lines 

=  [p'i,  Prid'd  a*,  ([cr]-1  (lp]R2))/u,  idA*]p)  by  substitution  definition 


by  substitution  definition 
2), 

by  previous  lines 
by  previous  lines 
by  substitution  definition 


Case  Ui  =  u[a]  and  U-2  =  u[t } 

A;  T  b  u[a]  :  Q'  by  assumption 

A;  T  b  cr  :  4r  and  A;  T  b  Q’  =  [c]Q  :  type  by  typing  inversion  lemma  12 

A  =  A1,u::'S/hQ,  A2 

A;  T  b  u[t\  :  Q'  by  assumption 

A;  T  b  r  :  4c  and 

A;  T  b  Q'  =  [ t]Q  :  type  by  typing  inversion  lemma  12 
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A;  T  h  [a]Q  =  [t\Q  :  type  by  transitivity 

A;  T  h  t  fl  a  :  T  'I/2  by  lemma  58 

(A^-uc^hQ,  A2);T  lb  u[a]  =  u[t]  /  ((Ax,  v::^2^Q,  A'2),  (idAl,  v[\dq,2]/u,  idA2)) 
with  A'2  =  |idAl,u/[idvp2]/n]A2  by  rule 

9  =  (9i,U/u,  62)  and  A;  T  b  U  :  Q  by  assumption 

[0](rt[cr])  =  [d](-u[r])  by  assumption 

[a]U  =  [ t]U  by  substitution  definition 

([id<j2]C/)  =  U  by  definition 

let  be  p'  =  ( 9\ ,  U / v ,  d2 ) 

9  =  [p/](idAl,n[idvp2]/'u,idA2) 


□ 


The  main  efficiency  problem  in  pattern  unification  lies  in  treating  the  case  for 
existential  variables.  In  particular,  we  must  traverse  the  term  U.  First  of  all,  we  must 
perform  the  occurs-check  to  prevent  cyclic  terms.  Second,  we  may  need  to  prune  the 
substitutions  associated  with  existential  variables  occurring  in  U.  Third,  we  need  to 
ensure  that  all  bound  variables  occurring  in  U  do  occur  in  the  range  of  a,  otherwise 
[up1  U  does  not  exist. 

In  the  next  section,  we  will  show  how  linearization  can  be  used  to  enforce  the  two 
criteria  which  eliminates  the  need  to  traverse  U.  First,  we  will  enforce  that  all  ex¬ 
istential  variables  occur  only  once,  thereby  eliminating  the  occurs-check.  Second,  we 
will  require  that  the  substitution  a  associated  with  existential  variables  is  always  a 
permutation  it,  thereby  restricting  higher-order  patterns  further.  This  ensures  that 
the  substitutions  are  always  invertible  and  eliminates  the  need  for  pruning.  Restrict¬ 
ing  higher-order  patterns  even  further  to  patterns  where  existential  variables  must  be 
applied  to  all  bound  variables  has  also  been  used  by  Hanus  and  Prehofer  [26]  in  the 
context  of  simply-typed  higher-order  functional  logic  programming.  While  Hanus  and 
Prehofer  syntactically  disallow  terms  which  are  not  fully  applied,  we  translate  any 
term  into  a  linear  higher-order  pattern  thereby  factoring  out  and  delaying  subterms 
which  are  not  applied  to  all  bound  variables.  The  resulting  assignment  algorithm  is  a 
specialization  of  the  presented  higher-order  pattern  unification  algorithm. 
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3.2  Rules  for  higher-order  pattern  unification 

A;  r,  x:A  b  Ux  =  U2  /  (A',  9)  A;  T  lb  Rx  =  R2  /  (A',  9) 

lam  .  „  . — - — : — - — ,  7 .  ,  coerce 


A;  T  h  Xx-.A.Ux  =  Xx:A.U2  /  (A',  9) 


A;  r  b  Rx  =  R2/  (A',  9) 


A;T  b  i?  |  [a]”1  =*(A *,  p)  A*  =  (AJ,  u::\p]^\p]Q,  A*) 
u  does  not  occur  in  R  A'2  =  |idA*,  [a]-1  (\p\R)/u.  id^*] A^ 
(A^'uu'FhQ,  A2);T  lb  u[a\  =  i?/((A^,A'2),  |idA*,  [a]-1  (\pjR)/u,  idA*]p) 

A;T  b  i?  |  [a]"1  =>(A\p)  A*  =  {Al,u::\p]^\p]Q,A^) 
u  does  not  occur  in  R  A2  =  [idA*,  [cr]_1  ([ p}R)/u,  idA*]A2 
(Ai,w::^i-Q,  A2);T  lb  R  =  u[a]  /  ((A*,  A2),  |idA*,  [a]-1  (\pjR)/u,  idA*]p) 


exl 


ex2 


(Ai,  u::^\-Q,  A2);  T  b  r  D  a  :  d'  =>  \H2  A2  =  [idAl,  v'[\dy2]/v}A2 
(Ai,  u::^\-Q,  A2);  T  lb  u[a\  =  u[r\  /  ((Ai,  v::'5/2\-Q,  A2),  (idAl,  n[id^2]/rt,  idAJ) 


ex3 


A;  r  lb  R1  =  R2  /  (Ai,  9 0  Ai;  [0i]r  b  =  [^]C/2  /  (A2,  02) 
A; rib  i?!  C/i  =  R2U2/(A2,  [6>i1(6>2)) 


app 


A;  T  lb  a;  =  a;  /  (A,  idA) 


var 


A;  T  lb  c  =  c  /  (A,  idA) 


const 


3.3  Linearization 

One  critical  optimization  in  unification  is  to  perform  the  occurs-check  only  when  nec¬ 
essary.  While  unification  with  the  occurs-check  is  at  best  linear  in  the  sum  of  the  sizes 
of  the  terms  being  unified,  unification  without  the  occurs-check  is  linear  in  the  smallest 
term  being  unified.  In  fact  the  occurs-check  can  be  omitted  if  the  terms  are  linear,  i.e., 
every  existential  variable  occurs  only  once. 

Let  us  consider  a  program  which  evaluates  expressions  from  a  small  functional 
language  Mini-ML.  First,  we  give  the  grammar  for  expressions. 


expressions  e  ::=  z  |  s  e  |  lam  x.e  |  (app  e\  e2) 
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To  describe  that  expression  e  evaluates  to  some  value  v,  we  use  the  following  judg¬ 
ment:  e  v. 


e  v 

z  ^  z  (s  e)  (s  v)  lam  x.e  lam  x.e 

e\  lam  x.e!  e2  v2  [v2/x\e!  >  v 

app  ei  e2  ^  u 

The  grammar  of  expressions  and  the  evaluation  judgment  are  represented  in  pure 
LF  the  following  way.  For  a  more  detailed  general  introduction,  we  refer  the  reader  to 
[49], 


exp  :  type. 

z  :  exp. 

s  :  exp  — > 

exp. 

app  :  exp 

— > exp  - 

■>  exp. 

lam  :  (exp 

-»■  exP) 

— ■>  exp 

eval  :  exp 

->  exp  — i 

>  type. 

Next,  we  represent  the  evaluation  inference  rules  in  pure  LF.  We  concentrate  here 
on  the  rule  for  functions. 

evJam  :  Lle:(exp  — >•  expj.eval  (lam  (Aaxexp.e  x))  (lam  (Aaxexp.e  a;)). 

The  variable  e  is  quantified  outside  with  a  fl.  When  executing  this  signature  as 
a  logic  program,  we  view  the  variable  e  as  a  logic  (existential)  variable,  which  will 
be  instantiated  using  unification  during  the  execution  of  the  logic  program.  A  more 
detailed  description  of  the  operational  semantics  and  interpreting  LF  types  as  logic 
programs  is  given  later  in  Chapter  4.  Here  we  just  note  that  existential  variables  arise 
during  logic  programming  execution  and  need  to  be  instantiated  via  unification.  Note 
that  e  has  function  type  exp  — »  exp.  The  instantiation  for  e  will  be  some  A-abstraction 
where  x  needs  to  be  applied  to.  /3-reduction  needs  to  be  applied  to  reduce  the  resulting 
expression. 
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The  modal  term  language  introduced  earlier  in  Chapter  2,  allows  a  more  compact 
description. 


evJam  :  IIDe::(|/:exphexp).eval  (lam  (Aaxexp ,e[x/y}))  (lam  (Ax:exp .e[x/y})). 

The  existential  variable  e  in  the  clause  evJam  is  quantified  by  IIDe::(|/:exphexp). 
Note  that  here  the  variable  e  is  already  lowered  and  has  atomic  type.  We  can  now  use 
just  first-order  replacement  (or  grafting)  to  instantiate  the  variable  e.  To  enforce  that 
every  existential  variable  occurs  only  once,  the  clause  head  of  evJam  can  be  translated 
into  the  linear  type 

nDe::(?/:exphexp).nne/::(^:exphexp).eval  (lam  (Ax:exp .e{x/y}))  (lam  (Aaxexp.  e'[x/z})) 
together  with  the  variable  definition 

Voxexp  .e'[x/z\  =  e[x/y}) 

where  e'  is  a  new  existential  variable.  Variable  definitions  are  defined  as  follows: 

Residuals  defining  variables  D  ::=  true  |  ti[id^]  =  U  \  D\  A  D2  \  \/x:A.D 

Then  we  can  use  a  constant  time  assignment  algorithm  for  assigning  a  linear  clause 
head  to  a  goal,  and  the  residuating  variable  definitions  are  solved  later  by  conventional 
unification.  As  a  result,  the  occurs-check  is  only  performed  if  necessary.  Note  that  the 
nn-quantifier  will  only  appear  at  the  outside,  and  we  can  view  the  clause  in  IP-form 
as  the  compiled  form  of  the  original  pure  LF  clause. 

In  the  dependency  typed  lambda-calculus,  there  are  several  difficulties  in  perform¬ 
ing  this  optimization.  First  of  all,  all  existential  variables  carry  their  context  T  and 
type  A.  If  we  introduce  a  new  existential  variable,  then  the  question  arises  what  type 
should  be  assigned  to  it.  As  type  inference  is  undecidablc  in  the  dependency  typed 
case,  this  may  be  expensive.  In  general,  we  may  even  obtain  a  term  which  is  not 
necessarily  well-typed. 

To  illustrate,  let  us  modify  the  previous  example,  and  annotate  the  expressions 
with  their  type  thus  enforcing  that  any  evaluation  of  a  Mini-ML  expression  will  be 
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well-typed.  First,  we  give  the  grammar  for  Mini-ML  types  which  includes  the  natural 
numbers  nat  and  the  function  type  t\  — »  f2. 

Mini-ML  types  t  ::=  nat  |  tx  — >  t2 

Next,  we  give  typing  rules  for  Mini-ML  expressions.  The  main  typing  judgment  is 
the  following: 

The:! 

which  means  the  Mini-ML  expression  e  has  Mini-ML  type  t  in  the  typing  context 
T.  The  typing  rules  are  straightforward: 

The:  nat 

Thz:  nat  T  h  (s  e)  :  nat 

T,  x:ti  he:f2  T  h  e\  :  t2  — >  t  T  h  e2  :  t2 

T  h  lam  x.e  :  ti  — >  f2  T  h  (app  e\  e2)  :  t 

To  allow  for  Mini-ML  types  also  in  our  LF  specification,  we  declare  a  type  family 
tp  for  Mini-ML  types,  which  has  as  elements  the  Mini-ML  type  nat  and  — 

tp  :  type. 

nat  :  tp. 

arr  :  tp  — >  tp  — » tp. 

Instead  of  implementing  the  typing  rules  for  Mini-ML  separately  to  the  evaluation 
rules,  we  annotate  the  expressions  with  their  corresponding  Mini-ML  types  by  using  the 
dependent  types  available  in  LF.  The  dependent  types  then  ensures  that  any  evaluation 
of  Mini-ML  expressions  is  well-typed.  This  has  the  advantage  that  a  type-preservation 
theorem  does  not  need  to  be  established  separately.  We  proceed  as  follows.  The  type 
family  exp  for  Mini-ML  expressions  is  indexed  by  Mini-ML  types  tp. 

exp  :  tp  — >  type. 

z  :  exp  nat. 

s  :  exp  nat  — >  exp  nat. 

app  :  nt1:tp.IIt2:tp.exp  (arr  t\  t2)  — > ►  exp  tx  — >  exp  f2. 

lam  :  nt1:tp.IIt2:tp.((exp  ti)  — >  (exp  f2))  — >  (exp  (arr  tx  f2)). 
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The  LF-types  of  the  constants  z,  s  ,  app  ,  and  lam  directly  encode  the  previous 
Mini-ML  typing  rules  and  ensure  that  only  expressions  which  are  well-typed  according 
to  the  Mini-ML  typing  rules  can  be  stated.  In  the  modal  LF  calculus,  the  LF-types 
for  app  and  lam  are  written  as  follows. 

app  :  nn£i::(-htp).na£2::(-  ^  tp).exp  (arr  t2  t)  — >  exp  t2  — >  exp  t. 
lam  :  nDt1::(-htp).nnt2::(-^tp).((exp  t2)  — >  (exp  t ))  — >  (exp  (arr  t2  t )). 

In  the  following  discussion,  we  will  directly  write  the  LF  signatures  using  the  LF- 
notation.  The  evaluation  judgment,  now  takes  an  expression  of  Mini-ML  type  t  and 
returns  an  expression  of  the  same  type.  This  can  be  enforced  using  dependent  types 
of  LF  as  follows: 


eval  :  IInt::(-htp).exp  t  — >  exp  t  — >  type. 

Next,  we  reformulate  the  evaluation  rule  for  lambda-expressions  give  it  in  fully 
explicit  form. 

evJam  :  IInti::(-ntp).L[at2::(-ntp).L[ae:: (y/:exp  Hi-exp  t2). 

eval  (arr  t\  t2)  (lam  t\  t2  (Aaxexp  t\.e[x/y]))  (lam  t3  t2  (Ax:exp  t\.e[x/y])). 

Note  that  the  constructor  lam  is  indexed  by  the  Mini-ML  types  t\  and  t2.  Similarly 
eval  is  indexed  by  the  Mini-ML  type  (arr  ti  t2).  Due  to  these  dependencies,  we  now 
have  multiple  occurrences  of  t\  and  t2,  in  addition  to  the  duplicate  occurrence  of  e. 
During  linearization,  the  clause  head  of  evJam  will  now  be  translated  into 

nnti::(-htp).nDt2::(-i-tp).nDe::(r/:exp  t^exp  t2). 

nnt3::(-htp).IIDt4::(-htp).nnt5::(-htp).L[Dt6::(-htp).L[De,::(^:exp  ^i-exp  t2). 
eval  (arr  t\  t2)(lam  t3  t4  (Ax:exp  ti.e[x/y]))  (lam  t5  t6  (Ax:exp  t\.  e'[x/ z]))- 

and  the  following  residuals  for  defining  variables 

ti  =  t3  A  t\  =  f5  A  t2  =  A  t2  =  te  A  Vx:exp  t\.e'[x/z\  =  e[x/y] 

Note,  that  due  to  the  linearization,  the  linear  clause  head  is  clearly  not  well-typed. 
However,  the  clause  head  is  well-typed  after  carrying  out  the  substitution 

{ti/t3,  t1  /t5,  t2/t4,  t2/t6,e[z/y]/e'} 
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This  relies  on  the  fact,  that  we  can  find  an  appropriate  ordering  of  the  existen¬ 
tial  variables  in  A  such  that  dependencies  among  existential  variables  and  variable 
definitions  are  obeyed.  It  is  worth  pointing  out  that  these  complications  are  due  to 
dependent  types  in  our  framework  and  do  not  arise  in  the  simply  typed  case.  Note  that 
some  of  these  variable  definitions  are  in  fact  redundant,  which  is  another  orthogonal 
optimization  (see  [43]  for  an  analysis  on  a  fragment  of  LF). 

Another  approach  would  be  to  interpret  the  new  existential  variables  modulo  vari¬ 
able  definitions,  by  introducing  a  new  quantifier  hFw  =  U::^\-A  which  corresponds  to 
the  variable  definition  -ufidvj,]  =  U.  This  would  allow  us  to  type-check  clauses  which  may 
contain  variable  definitions  directly,  without  applying  the  variable  definitions.  Using 
this  new  notation  the  linear  clause  head  becomes 

nnfi::(-htp).nD  f2::(-i-tp).nne::(i/:exp  Ui-exp  t2)- 

irt3  =  ti::(-htp).fIDf4  =  t2::(-htp).Uat5  =  fi::(-htp).fIDf6  =  t2::(-\- tp). 

nDe'  =  e[z / y\::(z:ex p  Uhexp  t2). 

eval  (arr  U  t2)(larn  t3  f4  (Ax:exp  ti.e[x/y]))  (lam  t5  t6  (Ax:exp  f4.  e'[x/ z]))- 

Here,  we  do  not  pursue  this  idea  further,  since  the  variable  definitions  are  treated 
in  a  very  special  manner  in  the  implementation. 

The  idea  of  factoring  out  duplicate  existential  variables  can  be  generalized  to  replac¬ 
ing  arbitrary  subterms  by  new  existential  variables  and  creating  variable  definitions.  In 
particular,  the  process  of  linearization  also  replaces  any  existential  variables  v[a\  where 
a  is  not  a  permutation  by  a  new  variable  ufid^]  and  a  variable  definition  wfid^]  =  v[a\. 

Thus,  our  linear  term  language  can  be  characterized  as  follows: 

Normal  Linear  Objects  L  ::=  Arc: A  L  |  ■u[7Tr]|-P 
Neutral  Linear  Objects  P  ::=  c  |  x  \  PL 

We  assume  that  all  normal  linear  objects  are  in  canonical  form  and  all  existential 
variables  are  of  atomic  type.  As  shown  in  Chapter  2,  this  can  be  achieved  by  lowering 
and  raising  operations  which  are  justified  by  lemma  40.  The  linearization  itself  is  quite 
straightforward  and  we  will  omit  the  details  here.  In  the  actual  implementation,  we 
do  not  generate  types  A  and  contexts  T  for  the  new,  linearly  occurring  existential 
variables,  but  ensure  that  all  such  variables  are  instantiated  and  disappear  by  the  time 
the  variable  definitions  have  been  solved. 
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3.4  Assignment  for  higher-order  patterns 

In  this  section,  we  give  a  refinement  of  the  general  higher-order  pattern  unification 
which  exploits  the  presented  ideas.  The  algorithm  proceeds  in  three  phases.  First  we 
will  unify  a  linear  atomic  higher-order  pattern  L  with  a  normal  object  U.  Note  that 
U  may  not  be  linear  and  does  not  refer  to  any  variable  definitions.  During  the  first 
phase,  we  decompose  the  objects  L  and  U  and  assign  terms  to  the  existential  variables 
occurring  in  L.  The  following  judgments  capture  the  assignment  between  a  linear 
atomic  higher-order  pattern  L  and  a  normal  object  U .  We  write  9  for  simultaneous 
substitutions  p/i/ui, . . .  Un/unJ  for  existential  variables. 

A;  T  h  L  =  U/(6,  A',  E)  assignment  for  normal  objects 
A;  T  lb  P  =  R/ (9,  A',  E)  assignment  for  atomic  objects 

where  P  is  a  linear  neutral  object  and  L  is  a  normal  linear  object.  In  contrast, 
U  is  a  normal  object  and  R  is  a  neutral  object,  both  of  which  are  not  necessarily 
linear.  Similar  to  higher-order  pattern  unification,  the  assignment  algorithm  returns 
a  substitution  9  and  a  modal  context  A'.  In  addition,  we  may  generate  some  residual 
equations  E.  As  discussed  earlier,  linearization  of  a  term  may  in  general  lead  to 
linear  term  which  is  not  necessarily  dependency  typed.  However,  the  term  is  still 
approximately  well-typed  when  all  the  dependencies  among  types  are  erased  (for  more 
on  erasing  types  see  2).  Here,  we  assume  that  both  L  (resp.  P)  and  U  (resp.  R)  are 
approximately  well-typed  in  context  A  and  T,  i.e.  when  we  erase  the  type-dependencies 
then  L~  (resp.  P~) is  approximately  well-typed  in  the  erased  modal  context  A-  and 
erased  context  T~ .  Moreover,  approximate  typing  invariants  are  preserved  during  linear 
higher-order  pattern  unification. 

There  are  two  possible  kinds  of  residual  equations  we  need  to  solve  after  assignment 
succeeds,  residuals  due  to  variable  definitions  and  residuals  due  to  the  fact  that  one  of 
the  terms  in  the  unification  is  non-linear.  So  in  addition  to  the  residuals  for  defining 
variables,  we  define  the  residuals  E  as  follows: 

Residual  Equation  E  :=  true  U\  =  f/2  |  E\  A  E-2  \  \/x\A.E 

The  assignment  algorithm  itself  is  given  below  is  a  specialization  of  the  previous 
higher-order  pattern  unification  algorithm. 
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A;  T,  x:A  \-  L  =  U  /  (6,  A',  E) 

A;  T  h  Xx\A.L  =  \x:A.U  /  ( 6 ,  A',  Vx:A.E )  lam 


A;T  lb  P  =  R  /  (9,  A,  E) 


coerce 


A;  r  b  P  =  R  /  (9,  A,  E) 

A  =  (Ai,m::^i-(5,  A2)  A'2  =  |idAl,  [t]-1  f?/w]A2 
A;  T  h  «[7r]  =  i?  /  ((idAl,  [tt]-1  i2/«,  idAJ,  (Ax,  A'2),  true) 

A  =  (Ai,  A2) 


existsL 


A;  T  h  L  =  u[a\  /  (idA,  A,L  =  u[a ]) 
const 


existsR 


A;  T  lh  a;  =  a:  /  (idA,  A,  true 


A;  r  lh  c  =  c  /  (idA,A,true 


var 


A;  T  lh  P  =  R  /  (01}  Ai,  iA)  Ai;  [0x]r  h  [0JL  =  /  (02,  A2,  E2)  _ 

A;  r  lh  P  L  =  i?  U  /  ([02]0!,  a2,  e1  a  e2) 

Note  again  that  we  do  not  need  to  worry  about  capture  in  the  rule  lam,  since  ex¬ 
istential  variables  and  bound  variables  are  defined  in  different  contexts.  In  the  rule 
app,  we  give  the  same  rule  as  previously.  Applying  6i  to  the  second  premise  is  only 
necessary  to  maintain  the  invariant  that  the  resulting  substitution  6i  maps  all  exis¬ 
tential  variables  from  A  to  some  new  context  Ai.  Since  all  modal  variables  in  the 
linear  term  occur  uniquely,  6\  only  instantiates  modal  variables  from  P  and  has  no 
“effect”  on  the  object  L.  Note  that  the  case  for  unifying  an  existential  variable  u\n\ 
with  another  term  R  is  now  simpler  and  more  efficient  than  in  the  general  higher-order 
pattern  case.  In  particular,  it  does  not  require  a  traversal  of  R  (see  rule  existsL ).  Since 
the  inverse  of  the  substitution  n  can  be  computed  directly  and  will  be  total,  we  know 
[tt ]  1  R  exists  and  can  simply  generate  a  substitution  [t]-1  R/u.  Finally,  we  may  need 
to  postpone  solving  some  unification  problems  and  generate  some  residual  equations  if 
the  non-linear  term  is  an  existential  variable  (see  existsR). 

The  rules  for  solving  the  residual  equations  are  straightforward.  First,  the  rules  to 
solve  the  remaining  variable  definitions. 


A-Th  D/{A',6) 

The  variable  definitions  D  are  well-typed  in  the  context  V  and  modal  context  A. 
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If  the  variable  definitions  can  be  solved,  then  a  modal  substitution  9  is  returned,  s.t. 
A'  b  9  :  A.  Note,  that  we  assume  that  the  modal  substitution  9\,  which  is  the  result  of 
the  previous  assignment  algorithm,  has  been  already  applied,  when  we  use  the  following 
rules  to  solve  the  residual  variable  definitions. 


A;  r,  lb  D/( A',  9)  A;  T  lb  DJ{ Au  (fi)  Ai;  [0i]T  lb  [0i]D2/(A2, 92) 


A;  r  lb  Vx:A.D/(A',  9)  A;  r  lb  D1  A  D2/( A2,  J^]^) 

_ A!2  =  [idAl,  U/u}A2 _ 

Aj,  u::(Tm4),  A2;  T  lb  u[  idr]  =  U/(  A1?  A'),  (idAl,  U/u,  idAJ) 


A;  r  b  [/'  =  U/{A',9) 
A;  r  lb  U'  =  U/{A',0) 


A;  T  lb  true/ (A,  idA) 


Solving  variable  definitions  is  then  straightforward.  The  only  case  we  need  to  pay 
attention  is  the  rule  (*).  This  case  may  arise  if  u[idr]  is  a  modal  variable  which  has  not 
been  instantiated  yet  during  assignment.  In  this  case,  we  simply  assign  u  the  object  U . 
This  ensures  that  after  the  variable  definitions  are  solved,  no  modal  variables  which  are 
used  in  variable  definitions  are  left  uninstantiated.  In  other  words,  all  the  intermediate 
modal  variables  which  have  been  introduced  during  linearization  are  now  instantiated. 

Next,  we  show  the  rules  for  solving  the  residual  equations,  which  have  been  gener¬ 
ated  during  assignment.  The  algorithm  is  straightforward  and  very  similar  to  solving 
variable  definitions,  except  that  no  special  provisions  have  to  be  made. 


A;T  fE  /  (A',0) 


The  residual  equations  E  are  well-typed  in  the  context  T  and  modal  context  A. 
If  the  residual  equations  can  be  solved,  then  a  modal  substitution  9  is  returned,  s.t. 
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A'  b  6  :  A. 

A;  r,  a;:A  P  E/{ A',  6)  A;  V  P  E1/(Al,  6,)  Ai;  [0i]r  P  [0i]S2/(A2, 02) 

A;rPVx:A£/(A',0)  A;  T  P  ^  A  £2/(A2,  [02](0i)) 

A;  r  h  L  =  U/(A',9) 

A ;m  =  U/(A',0)  A;T  Ptrue/(A,idA) 

The  overall  algorithm  can  be  summarized  as  follows: 

1.  Higher-order  assignment:  A;  T  P  L  =  U/(9i,  Ai,  f?) 

2.  Solving  variable  definitions:  Ai;  •  P  [6b]-D/(#2,  A2) 

3.  Solving  residual  equations:  A2;  •  P  [[02]0i].E/(A3,  03) 

The  final  result  of  the  overall  algorithm  is  the  modal  substitution  [^sKU^y^i])- 
For  the  higher-order  pattern  fragment,  we  can  now  show  that  assignment  algorithm 
is  sound  and  complete  for  linear  higher-order  patterns.  In  principle,  it  is  possible  to 
generalize  these  statements  further  to  include  variable  definitions  and  residual  equations 
E  and  show  that  linearization  together  with  linear  higher-order  pattern  unification  is 
sound  and  complete.  This  would  requires  a  more  precise  description  of  the  linearization 
process  and  how  variable  definitions  can  be  eliminated.  In  particular  the  completeness 
theorem  would  be  more  complicated 

3.5  Experiments 

We  have  implemented  the  presented  algorithm  as  part  of  the  Twelf  system.  The  instan¬ 
tiation  is  applied  eagerly  by  using  destructive  updates.  As  mentioned  before,  lowering 
and  grafting  are  also  done  eagerly  using  destructive  updates.  In  this  section,  we  discuss 
some  experimental  results  with  different  programs  written  in  Twelf.  All  experiments 
are  done  on  a  machine  with  the  following  specifications:  1.60GHz  Intel  Pentium  Pro¬ 
cessor,  256  MB  cache.  We  are  using  SML  of  New  Jersey  110.0.3  under  Linux  Red  Hat 
7.1.  Times  are  measured  in  seconds.  In  the  tables  below,  the  column  “opt”  refers 
to  the  optimized  version  with  linearization  and  assignment,  while  the  column  “stand” 
refers  to  the  standard  implementation  using  general  higher-order  pattern  unification. 
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The  column  “trivial”  indicates  the  percentage  of  problems  which  are  already  linear  and 
have  no  residuating  variable  definitions.  The  column  “fail”  indicates  how  many  vari¬ 
able  definitions  were  not  solvable,  if  there  were  any.  The  last  column  gives  the  overall 
success  rate  with  respect  to  the  total  number  of  calls  to  assignment  made.  The  col¬ 
umn  “time  reduction”  refers  to  how  much  time  is  saved.  This  is  calculated  as  follows: 
(time M  —  timenew) /time aid-  In  addition,  we  give  the  factor  of  improvement,  which  can 
be  calculated  time0id/timenew3 . 

3.5.1  Higher-order  logic  programming 

In  this  section,  we  present  two  experiments  with  higher-order  logic  programming.  The 
first  one  uses  an  implementation  of  a  meta-interpreter  for  ordered  linear  logic  by  Po- 
lakow  and  Pfenning  [58].  In  the  second  experiment  we  evaluate  our  unification  al¬ 
gorithm  using  an  implementation  of  foundational  proof-carrying  code  developed  at 
Princeton  University  [2], 

Meta-interpreter  for  ordered  linear  logic 


example 

opt 

stand 

reduction 

in  time 

improve 

variabl 

trivial 

e  def 

fail 

assign 

succeed 

sqnt  (bf) 

0.84 

2.09 

60% 

2.49 

44% 

46% 

23% 

sqnt  (dfs) 

0.93 

2.35 

60% 

2.52 

44% 

47% 

22% 

sqnt  (perm) 

4.44 

7.11 

38% 

1.60 

44% 

52% 

20% 

sqnt  (rev) 

1.21 

1.70 

29% 

1.40 

45% 

48% 

21% 

sqnt  (mergesort) 

2.26 

3.39 

33% 

1.50 

46% 

53% 

20% 

As  the  results  for  the  meta-interpreter  demonstrate,  performance  is  improved  by 
to  a  factor  of  2.5.  Roughly,  45%  of  the  time  there  were  no  variable  definitions  at  all. 
From  the  non-trivial  equations  roughly  45%  were  not  unifiable.  This  means  overall,  in 
approx.  20%  -  30%  of  the  cases  the  assignment  algorithm  succeeded  and  the  failure  of 
unification  was  delayed.  It  is  worth  noting  that  77%  to  80%  of  the  calls  to  assignment 
presented  in  Section  3.4  fail  immediately. 

3Note  that  from  the  factor  of  improvement  one  can  compute  the  time-reduction  automatically. 
If  the  factor  of  improvement  is  k  then  this  results  in  1  —  (1  /k)  reduction  in  time,  since  ( time0id  — 
timenew)/time0id  =  1  -  (timenew / time0id)  =  1  -  (1/fc). 
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Recently,  Twelf  has  been  applied  to  mobile  code  security  in  the  foundational  proof- 
carrying  code  project  at  Princeton  [3].  To  provide  guarantees  about  the  behavior  of 
mobile  code,  programs  are  equipped  with  a  certificate  (proof)  that  asserts  certain  safety 
properties.  These  safety  properties  are  represented  as  higher-order  logic  programs. 
Twclf’s  logic  programming  interpreter  then  executes  the  specification  and  generates  a 
certificate  (proof)  that  a  given  program  fulfills  a  specified  safety  policy.  We  have  had 
the  opportunity  to  evaluate  the  presented  assignment  algorithm  using  programs  from 
the  proof-carrying  code  benchmark  provided  by  Andrew  Appel’s  group  at  Princeton 
University. 

Foundational  proof-carrying  code 


example 

opt 

stand 

reduction 

in  time 

improve 

variabl 

trivial 

e  def 

fail 

assign 

succeed 

inc 

5.8 

9.19 

36.7% 

1.58 

64% 

46% 

18% 

switch 

36.00 

49.69 

27.54% 

1.38 

64% 

48% 

19% 

mul2 

5.51 

9.520 

42.86% 

1.72 

64% 

46% 

18% 

div2 

121.96 

153.610 

20.63% 

1.26 

63% 

48% 

20% 

divx 

333.69 

1133.150 

70.50% 

3.39 

63% 

50% 

21% 

listsum 

1073.33 

oo 

oo 

oo 

65% 

45% 

18% 

polyc 

2417.85 

oo 

oo 

oo 

65% 

41% 

17% 

pack 

197.07 

1075.610 

81.65% 

5.45 

66% 

45% 

19% 

In  the  table  above  we  show  the  performance  on  programs  from  the  proof-carrying 
code  benchmark.  Performance  is  improved  by  up  to  a  factor  of  5.45  and  some  ex¬ 
amples  are  not  executable  without  linearization  and  assignment.  The  results  clearly 
demonstrate  that  an  efficient  unification  algorithm  is  critical  in  large-scale  examples. 

There  are  several  reasons  for  this  substantial  performance  improvement.  1)  We 
reduced  the  need  for  trailing  instantiations  of  meta-variables  in  the  implementation. 
As  all  met  a- variables  in  the  clause  head  have  been  created  since  the  last  choice  point, 
no  trailing  should  be  necessary.  As  a  consequence,  less  space  is  needed  for  storing 
instantiation  of  variables.  However,  to  enforce  this  trailing  policy  in  a  high-level  imple¬ 
mentation  this  may  require  additional  bookkeeping  mechanism.  The  presented  simple 
assignment  algorithm  opened  another  possibility  to  reduce  the  need  for  trailing  with¬ 
out  modifying  the  actual  trailing  mechanism.  2)  We  not  only  factor  out  duplicate 
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meta-variables,  but  also  expressions  which  call  external  constraint  solvers.  Calling  the 
external  constraint  solvers  may  be  expensive,  especially  if  unification  fails  at  a  later 
point  anyway.  Factoring  out  constraint  expressions  and  delaying  their  solution,  allows 
us  to  fail  quickly  and  disregard  a  clause  as  not  applicable.  As  over  80%  of  the  clauses 
tried  fail,  this  is  particularly  important. 

Remark:  There  are  several  other  standard  optimizations  possible  such  as  for  ex¬ 
ample  first- argument  indexing.  Maybe  surprisingly,  first-argument  indexing  did  not 
improve  performance.  Although  the  number  of  calls  to  assignment  were  reduced  by 
roughly  10%,  this  did  not  seem  to  influence  the  overall  performance.  One  reason  we 
found  was  that  unification  and  weak  head  normal  form  dominate  the  run-time  dramat¬ 
ically. 

3.5.2  Higher-order  theorem  proving 

Besides  a  logic  programming  engine,  the  Twelf  system  also  provides  a  theorem  prover 
which  is  based  on  iterative  deepening  search.  In  this  section,  we  consider  two  exam¬ 
ples,  theorem  proving  in  an  intuitionistic  sequent  calculus  and  theorem  proving  in  the 
classical  natural  deduction  calculus. 

As  the  results  demonstrate,  the  performance  of  the  theorem  prover  is  not  greatly 
influenced  by  the  optimized  unification  algorithm.  The  main  reason  is  that  we  have 
many  dynamic  assumptions,  which  need  to  be  unified  with  the  current  goal.  However, 
we  use  the  standard  higher-order  pattern  unification  algorithm  for  this  operation  and 
use  the  optimized  algorithm  only  for  selecting  a  clause.  For  dynamic  assumptions  we 
cannot  maintain  the  linearity  requirement  and  linearizing  the  dynamic  assumptions  at 
run-time  seems  too  expensive. 


Proof  search  in  the  intuitionistic  sequent  calculus 


example 

opt 

stand 

reduction 

improve 

variable  def 

assign 

in  time 

trivial 

fail 

success 

dist-1 

53.00 

57.11 

7% 

1.08 

100% 

0% 

52% 

distlnrp 

0.40 

0.44 

9% 

1.10 

100% 

0% 

53% 

pierce 

1520.77 

1563.35 

3% 

1.03 

100% 

0% 

52% 

trans 

0.13 

0.13 

0% 

1.00 

100% 

0% 

53% 
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Proof  search  in  NK  (and,  impl,  neg) 


example 

opt 

stand 

reduction 

in  time 

improve 

variabk 

trivial 

3  def 

fail 

assign 

success 

andEffl-nk 

7.67 

13.14 

42% 

1.71 

100% 

0% 

80% 

andEff2-nk 

3.86 

6.58 

41% 

1.70 

100% 

0% 

81% 

assocAnd-nk 

2.24 

3.74 

40% 

1.67 

100% 

0% 

81% 

combS-nk 

3.85 

6.64 

42% 

1.72 

100% 

0% 

81% 

The  second  example  is  theorem  proving  in  the  natural  deduction  calculus.  In  con¬ 
trast  to  the  previous  experiments  with  the  sequent  calculus,  there  is  a  substantial 
performance  improvement  by  factor  1.7.  It  is  worth  pointing  out  that  in  both  ex¬ 
amples,  all  clause  heads  can  be  solved  by  the  optimized  unification  algorithm  and  no 
non-trivial  variable  definitions  arise.  Moreover,  most  of  the  time  unification  succeeds. 
This  is  not  surprising  when  considering  the  formulation  of  the  natural  deduction  sys¬ 
tem  as  any  elimination  rule  will  be  applicable  at  any  given  time.  Although  linear  head 
compilation  substantially  improves  performance,  more  optimizations,  such  as  tabling 
and  indexing,  are  needed  to  solve  more  complex  theorems. 

3.5.3  Constraint  solvers 

Finally,  we  consider  problems  using  constraint  solvers. 

Examples  using  constraint  solvers 


example 

opt 

stand 

reduction 

in  time 

improve 

variabl 

trivial 

e  def 

fail 

assign 

success 

solve 

1.73 

1.11 

-56% 

0.36 

7.8% 

42% 

82% 

represent 

0.16 

0.16 

0% 

1.00 

85% 

33% 

93% 

cont  Traction 

0.61 

0.64 

33% 

1.50 

79% 

12% 

82% 

mortgage 

4.88 

6.84 

29% 

1.40 

66% 

0% 

100% 

sieve  500 

2.61 

4.88 

46% 

1.86 

50% 

47% 

79% 

In  our  implementation,  we  delay  solving  any  sub-terms  which  call  constraint  solvers. 
As  the  results  below  demonstrate  this  may  be  beneficial  as  in  the  sieve  500  problem, 
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but  it  may  also  degrade  performance  in  rare  case  as  the  first  problem  demonstrates.  We 
left  out  the  reduction  of  time,  since  there  is  none  -  in  fact  time  increases.  This  has  been 
the  only  time  we  found  that  performance  deteriorated.  The  reason  for  performance 
degradation  in  this  example  is  that  most  unification  problems  were  outside  the  fragment 
of  linear  higher-order  patterns. 


3.6  Related  work  and  conclusion 

The  language  most  closely  related  to  our  work  is  AProlog.  Two  main  different  imple¬ 
mentations  of  AProlog  exist,  Prolog/Mali  and  Teyjus.  In  Prolog/Mali  implementation, 
the  occurs-check  is  left  out  entirely  [6].  While  Teyjus  [38,  37]  also  eliminates  some 
unnecessary  occurs-checks  statically  during  compilation.  However,  in  addition  to  the 
presence  of  dependencies  in  Twclf,  there  are  several  other  differences  between  our 
implementation  and  Teyjus.  1)  Teyjus  compiles  first  and  subsequent  occurrences  of 
existential  variables  into  different  instructions.  Therefore,  assignment  and  unification 
are  freely  mixed  during  the  execution.  This  may  lead  to  expensive  failure  in  some  cases, 
since  unification  is  still  called.  In  onr  approach,  we  perform  a  simple  fast  assignment 
check  and  delay  unification  entirely.  As  the  experimental  results  demonstrate,  only 
a  small  percentage  of  the  cases  fails  after  it  already  passed  the  assignment  test  and 
most  cases  benefit  from  a  fast  simple  assignment  check.  2)  We  always  assume  that 
the  types  of  existential  variables  are  lowered.  This  can  be  done  at  compile  time  and 
incurs  no  run-time  overhead.  In  Huet’s  unification  algorithm,  projection  and  imitation 
rules  are  applied  at  run-time  to  construct  the  correct  prefixes  of  A-abstractions.  3)  Our 
approach  can  easily  incorporate  definitions  and  constraint  domains.  This  is  important 
since  unifying  definitions  and  constraint  expressions  may  potentially  be  expensive.  In 
fact,  we  generalize  and  extend  the  idea  of  linearization  in  the  implementation  and  fac¬ 
tor  out  not  only  duplicate  existential  variables  but  also  any  difficult  sub-expressions 
such  as  definitions  and  constraint  expressions.  Therefore,  our  approach  seems  more 
general  than  the  one  adopted  in  Teyjus. 

As  experiments  show,  performance  is  improved  substantially.  This  is  especially 
important  in  large-scale  applications  such  as  proof-carrying  code  and  allows  us  explore 
the  full  potential  of  logical  frameworks  in  real-world  applications.  This  may  however 
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only  be  considered  as  a  first  step  towards  reducing  the  performance  gap  between  higher- 
order  and  first-order  system.  Another  optimization  which  is  particularly  important  to 
sustain  performance  in  large-scale  examples  is  term  indexing.  We  will  present  a  higher- 
order  term  indexing  technique  based  on  the  ideas  of  linearization  and  assignment  later 
in  chapter  5, 
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Chapter  4 

Tabled  higher-order  logic 
programming 


Tabled  first-order  logic  programming  has  been  successfully  applied  to  solve  complex 
problems  such  as  implementing  recognizers  and  parsers  for  grammars  [68] ,  representing 
transition  systems  CCS  and  writing  model  checkers  [16].  The  idea  behind  it  is  to 
eliminate  redundant  computation  by  memoizing  sub-computations  and  re-using  their 
results  later.  The  resulting  search  procedure  is  complete  and  terminates  for  programs 
with  the  bounded-term  size  property.  The  XSB  system  [62],  a  tabled  logic  programming 
system,  demonstrates  impressively  that  tabled  together  with  non-tabled  programs  can 
be  executed  efficiently. 

Higher-order  logic  programming  languages  such  as  Elf  [47]  extend  first-order  logic 
programming  in  three  ways:  first,  they  allow  the  programmer  to  define  his  or  her 
own  higher-order  data-types  together  with  constructors.  In  particular,  constructors 
may  take  functions  as  arguments  to  denote  the  scope  of  bound  variables.  Dependent 
types  allows  us  to  model  dependencies  among  the  constructor  arguments.  Second, 
we  may  generate  and  use  dynamic  assumptions  during  proof  search.  Third,  Elf’s  logic 
programming  engine  not  only  succeeds  or  fails,  but  also  returns  a  proof  giving  evidence 
why  it  succeeded. 

In  this  chapter,  we  give  a  motivating  example  and  explained  the  basic  idea  behind 
tabled  higher-order  logic  programming  where  some  redundant  computation  is  eliminated 
by  memoizing  sub-computations  and  re-using  their  result  later.  In  particular,  we  will 
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focus  on  some  of  the  higher-order  issues.  As  we  may  have  dynamic  assumptions  in 
higher-order  logic  programming,  goals  might  depend  on  a  context  of  dynamic  assump¬ 
tions.  We  also  have  dependencies  among  terms,  as  the  term  language  is  derived  from  the 
dependency  typed  A-calculus.  Finally,  in  contrast  to  tabled  first-order  logic  program¬ 
ming,  where  the  result  of  a  computation  is  an  answer  substitution  for  the  existential 
variables  in  the  query,  tabled  higher-order  logic  programming  needs  to  also  supply  the 
actual  proof  as  an  object  in  the  dependency  typed  lambda  calculus.  This  means  we 
need  to  store  in  the  memo-table  not  only  answer  substitutions  to  memoized  goals,  but 
also  the  actual  proof.  The  combination  of  these  three  features  requires  careful  design 
of  the  table  and  table  operations  and  poses  several  challenges  in  implementing  a  tabled 
higher-order  logic  programming  interpreter. 


4.1  A  motivating  example:  subtyping 

As  a  running  example  we  consider  a  type  system  for  a  restricted  functional  language 
Mini-ML,  which  includes  subtyping.  We  include  an  example  datatype  bits,  along  with 
two  subtypes  nat  for  natural  numbers  (bit-strings  without  leading  zeroes),  pos  for 
positive  numbers,  and  zero  for  the  number  zero. 

types  A  ::  =  zero  |  pos  |  nat  |  bits  |  A\  =>•  A2  \  . . . 

expressions  M  ::=  e  |  M  0  |  M  1  |  lam  x.M  j  app  Mi  M2 

We  represent  natural  numbers  as  bit-strings  in  standard  form,  with  the  least  signif¬ 
icant  bit  rightmost  and  no  leading  zeroes.  We  view  0  and  1  as  constructors  written  in 
postfix  form  and  e  stands  for  the  empty  string.  For  example  6  is  represented  as  e  1  10. 
Next,  we  give  the  subtyping  relation  for  types,  including  reflexivity  and  transitivity. 

- zn  - pn  - nb 

zero  nat  pos  ^  nat  nat  ■<  bits 


- refl 

A  -<  A 


A<B  B  <C 

- tr 

A  ~<  C 


The  typing  rules  our  Mini-ML  language  are  described  as  follows: 
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The:  bits 

r  b  M  : 

pos 

rb  M  0 

:  pos 

r  b  M  : 

nat 

rb  M  1 

:  pos 

T  b  M  :  A 


tp_sub 


tp^zl 

T  b  M:  bits 
- tp_z2 

r  b  M  0  :  bits 

tp_ol 

rb  M:  bits 
- tp_o2 

TbM  1  :  bits 

A'  ~<  A 

—  .  i 

r,  x\A\  b  M  :  A2 

T  b  lam  x.M  :  A\  =>■  A2 


tp_lam 


The  subtyping  relation  is  directly  translated  into  Elf  using  logic  programming  no¬ 
tation.  We  first  declare  a  type  family  tp  for  Mini-ML  types  with  constants  bit,  zero, 
pos,  and  nat  represent  the  basic  types  and  the  function  type  is  denoted  by  T1  =>  T2. 
Similarly,  we  declare  a  type  family  exp  for  Mini-ML  expressions. 


tp 

:  type . 

exp 

:  type . 

zero 

:  tp. 

e 

:  exp . 

pos 

:  tp. 

0 

:  exp  ->  exp. 

bits 

:  tp. 

1 

:  exp  ->  exp. 

=> 

:  tp  ->  tp  ->  tp. 

lam 

:  (exp  ->  exp)  ->  exp 

app 

:  exp  ->  exp  ->  exp. 

To  ease  readability,  we  assume  the  function  type  =>  is  declared  as  an  infix  operator, 
and  the  constructors  0  and  1  are  declared  as  postfix  operators. 

Next,  we  give  an  implementation  of  the  subtyping  relation  and  the  typing  rules  as 
higher-order  logic  programs.  Throughout  this  example,  we  reverse  the  arrow  Ai  — >  A2 
writing  instead  A2  <—  A1.  From  a  logic  programming  point  of  view,  it  might  be  more 
intuitive  to  think  of  the  clause  H  <—  Ai  An  as  H  <—  A\  ,  ...  ,  An. 


sub 

:  tp  ->  tp  ->  type. 

ref  1 

:  sub  A  A. 

zn 

:  sub  zero  nat . 

nb 

:  sub  nat  bits. 

pn 

:  sub  pos  nat . 

tr 

:  sub  A  C 

arr 

:  sub  (Al  =>  A2) 

<-  sub  A  B 

<-  sub  B1  Al 

<-  sub  B  C . 

<-  sub  A2  B2 . 
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of 

tp_sub 


exp  ->  tp  ->  type. 


of  M  A 
<-  of  M  A’ 

<-  sub  A’  A. 


tp_lam 


tp_app 


of 

(lam  ( [x]  M  x))  (A1  => 

A2) 

<- 

<{* 

::exp}  of  x  A1  ->  of 

(M  x) 

A2)  . 

of 

(app  Ml  M2)  T 

<- 

of 

Ml  (T2  =>  T) 

<- 

of 

M2  T2 . 

etc. 

are 

implicitly  quantified  i 

at  the 

outside 

Twelf  internally  reconstructs  the  types  of  the  logic  variables  A,  B,  etc.  In  LF,  the 
transitivity  clause  is  written  as  follows: 


tr  :  fla:tp.n6:tp.ric:tp.sub  b  c  — »  sub  a  b  — >  sub  a  c. 

Under  the  type-theoretic  view,  na:tp.n&:tp.IIc:tp.sub  b  c  — >  sub  a  b  sub  a  c  is  the 
type  of  the  constant  tr.  Under  the  logic  programming  interpretation,  we  interpret  types 
as  formulas  and  we  refer  to  Ila:tp.n&:tp.nc:tp.sub  b  c  — >  sub  a  b  — >  sub  a  c  as  a  clause 
and  tr  as  the  name  of  the  clause,  and  In  the  following  presentation,  we  often  use  types 
and  formulas  interchangeably. 

The  proof  for  zero  y<  bits  which  is  given  by  the  following  derivation 


-  ZTi  - 

zero  y<  nat  nat  A  bits 


rib 

tr 


is  represented  as  a  proof  term  (tr  zero  nat  bits  nb  zn)  in  LF.  The  constant  tr  is  a 
function  where  the  existential  variables  a,  b,  c  are  instantiated  to  zero,  nat,  bits.  In 
addition  we  pass  in  as  arguments  the  proof  term  nb  for  nat  y<  bits  and  the  proof  term  zn 
for  zero  y<  nat.  Note  that  we  usually  omit  the  implicit  arguments  zero  and  nat,  which 
denote  the  instantiation  of  transitivity  rule  and  just  write  (tr  nb  zn)  for  the  proof  of 
zero  ■<  bits.  In  Twelf  notation,  this  corresponds  to  the  proof  term  (tr  bn  zn)  for  the 
proof  of  the  query  (sub  zero  bit). 

Although  the  specification  and  implementation  of  the  typing  rules  including  sub¬ 
typing  is  straightforward,  the  implementation  is  not  directly  executable.  Computation 
may  be  trapped  in  infinite  paths  and  performance  may  be  hampered  by  redundant 
computation.  For  example,  the  execution  of  the  query  sub  zero  T  will  end  in  an 
infinite  branch  trying  to  apply  the  transitivity  rule.  Similarly,  the  execution  of  the 
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query  of  (lam  [x]  x)  T  will  not  terminate  and  fail  to  enumerate  all  possible  types. 
In  addition,  we  repeatedly  type-check  sub-expressions,  which  occur  more  than  once. 
To  eliminate  redundancy,  some  sophisticated  type  checkers  for  example  for  refinement 
types  memoize  the  result  of  sub-computations  to  obtain  more  efficient  implementa¬ 
tions.  In  this  chapter,  we  extend  higher-order  logic  programming  languages  such  as 
Elf  with  generic  memoization  techniques,  called  tabled  higher-order  logic  programming. 
This  has  several  advantages.  Although  it  is  possible  to  derive  an  algorithmic  subtyping 
relation  for  the  given  example,  this  might  not  be  trivial  in  general.  One  could  argue 
that  the  user  can  refine  the  implementation  further  and  add  an  extra  argument  to  the 
type  family  of,  which  can  be  used  to  store  intermediate  results,  thereby  developing  a 
customized  type-checker  with  explicit  support  for  memoization.  However,  this  com¬ 
plicates  the  type  checker  substantially  and  proving  the  correctness  of  the  type-checker 
with  special  memoization  support  will  be  hard,  because  we  need  to  reason  explicitly 
about  the  structure  of  memoization.  Moreover,  the  certificates,  which  are  produced  as 
a  result  of  the  execution,  are  larger  and  contain  references  to  the  explicit  memoization 
data-structure.  This  is  especially  undesirable  in  the  context  of  certified  code  where 
certificates  are  transmitted  to  and  checked  by  a  consumer,  as  sending  larger  certifi¬ 
cates  takes  up  more  bandwidth  and  checking  them  takes  more  time.  Finally,  tabled 
logic  programming  provides  a  complete  proof  search  strategy.  For  programs  with  the 
bounded-term  size  property,  tabled  logic  programming  terminates  and  we  are  able  to 
disprove  certain  statements1.  This  in  turn  helps  the  user  to  debug  the  specification 
and  implementations. 


4.2  Tabled  logic  programming:  a  quick  review 

Tabling  methods  evaluate  programs  by  maintaining  tables  of  subgoals  and  their  an¬ 
swers  and  by  resolving  repeated  occurrences  of  subgoals  against  answers  from  the  table. 

1The  size  of  a  term  is  defined  recursively  where  the  size  of  a  bound  or  modal  variable  or  constant  is 
1  and  the  size  of  R  U  is  the  size  of  R  plus  the  size  of  XJ.  In  the  higher-order  setting,  we  can  in  addition 
define  the  size  of  Xx:A.U,  as  1  plus  the  size  of  U  and  assume  that  all  terms  must  be  eta-expanded.  A 
finite  program  P  has  the  bounded-term  size  property  if  an  atomic  query  Q  has  no  arguments  whose 
size  exceeds  n  and  there  is  a  function  /(n),  then  there  is  no  atomic  subgoal  in  the  derivation  whose 
an  argument  size  exceeds  f(n).  (see  [12],  section  5.4). 
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We  review  briefly  Tamaki  and  Sato’s  multi-stage  strategy  [65],  which  differs  only  in¬ 
significantly  from  SLG  resolution  [12]  for  programs  without  negation.  To  demonstrate 
tabled  computation,  we  consider  the  evaluation  of  the  query  sub  zero  T  in  more  detail. 

The  search  proceeds  in  multiple  stages.  The  table  serves  two  purposes:  1)  We 
record  all  sub-goals  encountered  during  search.  If  the  current  goal  is  not  in  the  table  or 
more  precisely  there  is  no  entry  A  s.t.  A  is  an  alpha-variant  of  the  current  goal,  then 
we  add  it  to  the  table  and  proceed  with  the  computation.  We  consider  a  goal  A  an 
alpha-variant  of  another  goal  A'  if  there  exists  a  bijection  between  the  modal  variables 
and  ordinary  variables. 

Computation  at  a  node  is  suspended,  if  the  current  goal  is  a  variant  of  a  table  entry. 
2)  In  addition  to  the  sub-goals  we  are  trying  to  solve,  we  also  store  the  result  of  the 
computation  in  the  table  as  a  list  of  answers  to  the  sub-goals.  An  answer  is  a  pair  of 
a  substitution  6  for  the  existential  variables  in  the  current  goal  A  together  with  the 
actual  proof  P. 

In  each  stage  we  apply  program  clauses  and  answers  from  the  table.  Figure  4.1 
illustrates  the  search  process. 

The  root  of  the  search  tree  is  labeled  with  the  goal  sub  zero  A.  Each  node  is  labeled 
with  a  goal  statement  and  each  child  node  is  the  result  of  applying  a  program  clause 
or  an  answer  from  the  table  to  the  leftmost  atom  of  the  parent  node.  Applying  a 
clause  H  <—  Ai  <—  A2 . . .  <—  An  results  in  the  subgoals  A1,A2, ...  ,An  where  all  of 
these  subgoals  need  to  be  satisfied.  We  will  then  expand  the  first  subgoal  Ai  carrying 
the  rest  of  the  subgoals  A2, ...  ,An  along.  If  a  branch  is  successfully  solved,  we  show 
the  obtained  answer.  To  distinguish  between  program  clause  resolution  and  re-using 
of  answers,  we  have  two  different  kinds  of  edges  in  the  tree.  The  edges  obtained  by 
program  clause  resolution  are  solid  annotated  with  the  clause  name  used  to  derive 
the  child  node.  Edges  obtained  by  reusing  answers  from  the  table  are  dashed  and 
annotated  with  the  answer  used  to  derived  the  child  node.  Using  the  labels  at  the 
edges,  we  can  reconstruct  the  proof  term  for  a  given  query.  In  general,  we  will  omit 
the  actual  substitution  under  that  the  parent  node  unifies  with  the  program  clause  to 
avoid  cluttering  the  example.  To  ensure  we  generate  all  possible  answers  for  the  query, 
we  restrict  the  re-use  of  answers  from  the  table.  In  each  stage,  we  are  only  allowed  to 
re-use  answers  that  were  generated  in  previous  stages.  Answers  from  previous  stages 
(available  for  answer  resolution)  are  marked  gray,  while  current  answers  (not  available 
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Stage  1 


Stage  2 


sub  zero  A 
refl 


A  =  zero 


(1)  ;--»A=zero 

(2)  J-»A=  nat 

subzero*,; 

sub  R  A.  sub  zeroA  - 

»  sub  nat  A 


(3) 


-  A  =  nat 


Entry 

Answers 

sub  zeroA 

A  =  zero:  refl  (1) 
A  =  nat  :  zn  (2) 

tr 

'  a  =  nai 

'  sub  nat  R , 

(4) 

nb 

(5) 

(3) 

Entry 

Answers 

Ei 

sub  zeroA  A=zero:refl  (1) 
A  =nat  :  zn  (2) 
A  =bits  :  (tr  bn  zn)(3) 


sub ; 


sub  nat  A 


A  =nat  :  refl 
A  =bits  :  nb 


sub  i 
sub 


Figure  4.1:  Staged  computation 


yet)  are  black.  In  the  first  stage  no  answers  are  available  yet.  In  the  second  stage,  we 
are  only  allowed  to  use  answers  (1)  and  (2).  In  the  third  stage,  we  may  use  all  answers 
from  (1)  up  to  (5).  In  the  final  stage,  we  are  allowed  to  use  all  answers. 


4.3  Tabled  higher-order  logic  programming 

In  tabled  higher-order  logic  programming,  we  extend  tabling  to  handle  sub-goals  that 
may  contain  implications  and  universal  quantification  and  our  term  language  is  the 
dependently  typed  A-calculus.  The  table  entries  are  no  longer  atomic  goals,  but  atomic 
goals  A  together  with  a  context  T  of  assumptions.  In  addition,  terms  might  depend 
on  assumptions  on  T.  To  highlight  some  of  the  challenges  we  present  the  evaluation  of 
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Stage  1 


of  (lam  [x]  x)  A 

tp-sub  _  of  (lam  [x]  x)  B. 
sub  B  A 


lpJam  -  u:of  x  At 


of  x  A2 


u 


tp_sub 


A1  =  T,  A2  =  T,  A  =  T  =>  T 

u:ofxA1h  of  x  B. - 

sub  B  A2 


Entry 

Answers 

of  (lam  [x]  x)  A 

A  =  T  =>  T  :  (tpjam  [u]  u)  (1) 

u:ofxA1h  0fxA2 

AI  =  T,  A2=  T  :  u  (2) 

Figure  4.2:  Staged  computation  for  identity  function 
the  query  of  (lam  [x]  x)  T  in  Figure  4.2. 

The  possibility  of  nested  implications  and  universal  quantifiers  adds  a  new  degree 
of  complexity  to  memoization-based  computation.  Retrieval  operations  on  the  table 
need  to  be  redesigned.  One  central  question  is  how  to  look  up  whether  a  goal  T  h  P  is 
already  in  the  table.  There  are  two  options:  In  the  first  option  we  only  retrieve  answers 
for  a  goal  P  given  a  context  T,  if  the  goal  together  with  the  context  matches  an  entry 
T'  h  P'  in  the  table.  In  the  second  option  we  match  the  subgoal  P  against  the  goal 
P'  of  the  table  entry  T'  h  P\  and  treat  the  assumptions  in  T'  as  additional  subgoals, 
thereby  delaying  satisfying  these  assumptions.  We  choose  the  first  option  of  retrieving 
goals  together  with  their  dynamic  context  T'.  One  reason  is  that  it  restricts  the  number 
of  possible  retrievals  early  on  in  the  search.  For  example,  to  solve  subgoal  w:of  x  Ai  b 
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of  x  _B,sub  B  A 2,  we  concentrate  on  solving  the  left-most  goal  u: of  x  A\  h  of  x  B 
keeping  in  mind  that  we  still  need  to  solve  u: of  x  A\  h  sub  B  A 22.  As  there  exists 
a  table  entry  w.of  x  A1  h  of  x  A2,  which  is  a  variant  of  the  current  goal  u:of  x  Ai  h 
of  x  B ,  computation  is  suspended.  Note,  we  adopted  a  common  convention  in  logic, 
where  we  omitted  the  explicit  declaration  of  the  parameter  x  in  the  context.  To  be 
more  precise,  we  write  in  the  following  development  axexp,  u: of  x  A\  for  the  context  of 
ordinary  variables  instead  of  just  u: of  x  A]_. 


4.4  Subordination 

Due  to  the  higher-order  setting,  the  predicates  and  terms  might  depend  on  assumptions 
in  the  context  T.  Virga  [67]  developed  in  his  PhD  thesis  techniques,  called  subordi¬ 
nation,  to  analyze  dependencies  in  Elf  programs  statically  before  execution.  In  the 
Mini-ML  example,  the  terms  of  type  exp  and  tp  are  independent  of  each  other.  On 
the  level  of  predicates,  the  type  checker  of  depends  on  the  subtyping  relation  sub,  but 
not  vice  versa.  When  checking  whether  a  subgoal  T  b  P  is  already  in  the  table,  we  ex¬ 
ploit  the  subordination  information  in  two  ways.  First,  we  use  it  to  analyze  the  context 
T  and  determine  which  assumptions  might  contribute  to  the  proof  of  P.  For  example 
the  proof  for  x:exp,u:of  x  A\  h  of  x  A2  depends  on  the  assumption  u.  However,  the 
predicate  sub  does  not  refer  to  the  predicate  of.  Similarly,  the  predicate  sub  does  not 
refer  to  expressions  exp.  Therefore  the  proof  for  axexp,  w:of  x  T  b  sub  A  A2  cannot  de¬ 
pend  on  the  assumption  u  or  x.  When  checking  whether  u: of  x  T  b  sub  T  A2  is  already 
in  the  table,  it  suffices  to  look  for  a  variant  of  sub  T  A2.  In  the  given  example,  compu¬ 
tation  at  subgoal  a;:exp,-u:of  x  T  b  sub  T  A2  is  suspended  during  stage  2  as  the  table 
already  contains  sub  Bi  T.  If  we  for  example  first  discover  axexp,  u:of  x  T  b  sub  T  A2, 
then  we  store  the  strengthened  goal  sub  T  A2  in  the  table  with  an  empty  context. 

Second,  subordination  provides  information  about  terms.  As  we  are  working  in  a 
higher-order  setting,  solutions  to  new  existential  variables,  which  are  introduced  during 
execution,  might  depend  on  assumptions  from  T.  For  example,  applying  the  subtyping 

2Note  that  u  in  the  declaration  it:of  x  Ai  denotes  an  ordinary  bound  variable,  not  a  modal  variable. 
Recall  that  to  denote  the  declaration  of  the  modal  variable  u  we  write  u\A>\-A 
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rule  to  oxexp,  w:of  x  A\  h  of  x  A2  yields  the  following  new  goal 

x:exp,  u:of  x  Ai  b  of  x  (B  x  u)  ,  sub  ( B  x  u )  A2 

where  the  solution  for  the  new  variable  B  might  depend  on  the  new  variable  axexp 
and  the  assumptions  w:of  x  A±.  However,  we  know  that  the  solution  must  be  an  object 
of  type  tp  and  that  objects  of  type  tp  are  independent  of  Mini-ML  expressions  exp  and 
the  Mini-ML  typing  rules  of.  Hence,  we  can  omit  x  and  u  and  write  x:exp,  u: of  x  A\  h 
of  x  B,  sub  B  A2.  Before  comparing  goals  with  table  entries  and  adding  new  table 
entries,  we  eliminate  unnecessary  dependencies  from  the  subgoal  T  h  P.  This  allows 
us  to  detect  more  loops  in  the  search  tree  and  eliminate  more  redundant  computation. 
We  come  back  to  this  issue  of  subordination  and  other  optimizations  to  detect  more 
loops  in  the  higher-order  setting  later  in  the  chapter  on  implementation  of  a  tabled 
higher-order  logic  programming  interpreter. 


4.5  A  foundation  for  tabled  higher-order  logic  pro¬ 
gramming 

In  this  section,  we  give  a  proof-theoretic  characterization  of  tabled  higher-order  logic 
programming  based  on  uniform  proofs  [36]  and  show  soundness  of  the  resulting  in¬ 
terpreter.  This  work  forms  the  basis  of  the  implemented  tabled  interpreter  for  the 
language  Elf.  Although  we  concentrate  on  the  logical  framework  LF,  which  is  the  basis 
of  Elf  it  seems  possible  to  apply  the  presented  approach  to  AProlog  [39]  or  Isabelle  [45], 
which  are  based  on  hereditary  Harrop  formulas  and  simply  typed  terms. 

4.5.1  Uniform  proofs 

Computation  in  logic  programming  is  achieved  through  proof  search.  Given  a  goal 
(or  query)  A  and  a  program  T,  we  derive  A  by  successive  application  of  clauses  of 
the  program  T.  Miller  et  al  [36]  propose  to  interpret  the  connectives  in  a  goal  A 


120 


4.5.  A  FOUNDATION  FOR  TABLED  HIGHER-ORDER  LOGIC  PROGRAMMING 


as  search  instructions  and  the  clauses  in  T  as  specifications  of  how  to  continue  the 
search  when  the  goal  is  atomic.  A  proof  is  goal-oriented  if  every  compound  goal  is 
immediately  decomposed  and  the  program  is  accessed  only  after  the  goal  has  been 
reduced  to  an  atomic  formula.  A  proof  is  focused  if  every  time  a  program  formula  is 
considered,  it  is  processed  up  to  the  atoms  it  defines  without  need  to  access  any  other 
program  formula.  A  proof  having  both  these  properties  is  uniform  and  a  formalism  such 
that  every  provable  goal  has  a  uniform  proof  is  called  an  abstract  logic  programming 
language. 

Elf  is  one  example  of  an  abstract  logic  programming  language,  which  is  based  on 
the  LF  type  theory.  Ax\Ai.A2  denotes  the  dependent  function  type  where  the  type 
A2  may  depend  on  an  object  x  of  type  A\.  Whenever  x  does  not  occur  free  in  A2  we 
may  abbreviate  Ax:A\.A2  as  A\  — >  A2.  Using  the  types  as  formulas  interpretation, 
we  refer  to  Ax:Ai.A2  as  the  universal  quantifier  and  treat  Ai  — >  A2  as  an  implication. 
Moreover,  the  typing  context  T  is  viewed  as  a  context  of  formulas  which  represents 
the  program.  We  will  use  types  and  formulas  interchangeably  in  the  presentation.  We 
give  the  fragment  used  for  logic  programming  in  LF  below.  Types  and  programs  are 
defined  as  follows: 

Types  A  ::=  P  \  Ai  — >  A2  \  Ax  :  Al.A2 
Programs  T  ::=  •  |  T,x:A 

P  ranges  over  atomic  formulas  i.e.  a  Mi  . . .  Mn.  The  clause 

tr:sub  A  C  <-  sub  A  B  <-  sub  B  C. 


is  interpreted  as 


tr:nt:tp.ns:tp.rir:tp.sub  rs->  (sub  t  r  — >  sub  t  s ) 

Every  type  (or  formula)  has  a  corresponding  proof  term  U .  We  assume  that  all 
proof  terms  are  in  canonical  form.  Recall  that  applications  which  are  in  canonical  form 
are  of  the  following  form:  (((H  Uf)  U2 ) . . .  Un)  where  head  PI  is  either  a  constant  c  or  a 
variable  x.  To  allow  direct  head  access,  we  flatten  the  representation  of  applications  and 
decompose  the  previous  application  into  its  head  c  and  the  arguments  U\,  U2,...Un. 
The  spine- notation  as  introduced  in  [11]  essentially  achieves  this  idea. 
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Terms  U  ::=  H  ■  S  \  Xx:A.U 
Spines  S  ::=  nil  |  U;  S 
Heads  H  ::=  c  \  x 

In  the  example  from  Sec.  4.1,  the  proof  term  corresponding  to  sub  zero  bits  is 
given  as  tr  zn  refl.  Note  that  we  actually  omitted  the  implicit  arguments  zero  and  nat, 
which  denote  the  instantiation  of  transitivity  rule.  In  the  following  discussion,  we  will 
include  implicit  arguments  in  the  proof  term  representation.  To  give  an  intuition  for 
this  notation  we  give  a  few  examples. 

(((((tr  zero)  nat)  bits)  nb)  zn) 

is  denoted  using  spine  notation  by 

tr  ■  (zero  ;  nat  ;  bits  ;  nb  ;  zn  ;  nil). 

The  proof  term 

((((tpJam  T)  (Ax:exp.x))  T)  (Ax:exp.Au:of  x  T.  u )) 

is  denoted  by 

tpJam  •  (T  ;  (Ax:exp.a:  •  nil)  ;  T  ;  (Aaxexp.Atcof  x  T.  u  ■  nil)  ;  nil). 

where  T  denotes  a  term  of  type  tp  for  example  the  term  (nat  •  nil).  We  can  charac¬ 
terize  uniform  proofs  by  two  main  judgments: 

T  — ^  U  :  A  U  is  a  uniform  proof  for  A  from  T 

f 

T  >  A  — >  S  :  P  S'  is  a  focused  proof  for  the  atom  P  by  focusing  on  clause  A 

Taking  a  type-theoretic  view,  we  can  interpret  the  first  judgment  as  U  has  type  A 
in  the  context  T  and  the  later  as  S  has  type  P  in  context  T.  Note  that  we  assume 
all  operations  are  capture-avoiding  and  we  can  rename  bound  variables  if  necessary. 
Inference  rules  describing  uniform  and  focused  proofs  are  given  in  Figure  4.3. 

In  the  rule  fV,  we  instantiate  the  bound  variable  x  with  a  term  U.  As  x  has  type  Ai, 
we  check  that  U  has  type  Ai  in  T.  The  proof  term  represents  the  witness  of  the  proof. 
When  searching  for  a  uniform  proof,  the  proof  term  is  constructed  simultaneously.  In 
the  following  discussion,  we  will  often  ignore  proof  terms,  but  keep  in  mind  that  they 
are  silently  generated  as  a  result  of  the  proof. 
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r,  X  :  A,  r  >  A  -U  S  :  P 

- - u_atom  - f_atom 

r  ,x  :  A,Y' ^  x  ■  S  :  P  r  »  P  nil  :  P 

T,x-.A^U:Az  r  »  [idr,  U/x]At  -L  S  :  P  ThU-.A, 

- - - - - uV  - ; - fV 

r  Ax  :  AX.U  :  Ux  :  AVA2  Y  >  Tlx  :  AX.A2  -UU\S:P 


r,c:  AXAUU  ■  A2 


r  ^>a1Ms:P 


U:A2 


f 


r — >A c:A1.U:A1^A2  r  >  A2  -»•  A^  U;  S  :  P 

Figure  4.3:  Uniform  deduction  system  for  C 


4.5.2  Uniform  proofs  with  answer  substitutions 

The  result  of  a  computation  in  logic  programming  is  generally  an  answer  substitution 
9  for  the  existentially  quantified  variables  in  a  goal  A.  To  obtain  an  algorithm  that 
computes  answer  substitutions,  we  substitute  existential  variables  u  for  the  bound 
variable  x  in  the  fV  rule.  We  will  interpret  the  existential  variables  as  modal  variables 
again  and  generalize  the  previous  judgment  to  include  a  context  A  of  modal  variables. 

Existential  variables  are  later  instantiated  later  during  unification  yielding  a  sub¬ 
stitution  9.  We  will  concentrate  here  on  the  case  of  higher-order  pattern  unification. 

There  are  different  alternative  approaches  to  treat  existential  variables.  One  we 
have  explored  in  [55]  relies  on  the  following  fact:  If  U  is  a  solution  for  x  in  the  con¬ 
text  T  then  there  exists  a  solution  U'  of  type  nr. Ax  such  that  U'  ■  Y  is  also  a  solu¬ 
tion  for  x  and  U'  is  well-typed  in  the  empty  context.  We  write  nr. Ax  for  the  type 
Y[x\\B\. . . .  Yxn:Bn.Ai  where  Y  is  a  context  x\:Bi, . . . ,  xn:Bn  and  U'  ■  Y  as  an  abbre¬ 
viation  for  U'  ■  x\\ . . .  "%xn\  nil.  Moreover,  there  is  a  one-to-one  correspondence  between 
these  two  solutions,  as  U'  ■  Y  reduces  to  U.  This  has  been  shown  by  Miller  [35]  for  the 
simply-typed  case  and  by  Pfenning  in  the  dependently  typed  and  polymorphic  case 
[48].  Following  Miller’s  terminology,  we  say  U'  is  the  result  of  raising  U .  Intuitively,  U 
depends  globally  on  the  assumptions  in  T.  Raising  allows  us  to  localize  dependencies 
by  replacing  U  with  U'  ■  T.  Then  we  can  translate  the  fV  rule  to 
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r  »  [idr,  u'  ■  T/x\A2  -U  s-  P  •  U'  :  UT.A1 

- fV 

r  >  Ux  :  A!.A2  {U'  •  r);  S  :  P 

To  obtain  a  calculus  with  existential  variables,  we  then  replace  U'  with  a  new 
existential  variable  which  is  annotated  with  its  type.  Another  alternative  would  be 
to  use  mixed-prefixes  [35]  to  model  dependencies.  However  this  would  complicate 
the  presentation  further.  Here,  we  use  the  modal  context  and  modal  substitution  as 
previously  introduced  in  chapter  2.  This  eliminates  the  need  of  first  translating  the 
fV-rule.  The  two  main  judgments  for  computing  answer  substitutions  are 

A;  T  AA  A/(A',  6)  Uniform  proof  with  answer  substitution  A'  b  9  :  A 

A;T>d  — ^  P/(  A',  6)  Focused  proof  with  answer  substitution  A'  b  6  :  A 

We  assume  that  A  is  well-typed  in  the  modal  context  A  and  bound  variable  context 
T.  The  inference  rules  are  given  in  Figure  4.4. 

To  obtain  an  algorithm,  we  impose  left-to-right  order  on  the  solution  of  the  fs  — » 
rule.  This  matches  our  intuitive  understanding  of  computation  in  logic  programming. 
In  the  fs  — >  rule  for  example  we  first  decompose  the  focused  clause  until  we  reach  the 
head  of  the  clause.  After  we  unified  the  head  of  the  clause  with  our  goal  A  on  the  right- 
hand  side  of  the  sequent  and  completed  this  branch,  we  proceed  proving  the  subgoals. 
This  left-to-right  evaluation  strategy  only  fixes  a  don’t  care  non-deterministic  choice  in 
the  inference  system.  In  the  fsV  rule  we  delay  the  instantiation  of  x  by  introducing  a 
new  modal  variable  rt [idr]  •  Note  that  this  variable  is  created  with  type  A  and  context  T. 
There  is  no  condition  needed  to  ensure  that  9  does  not  leave  its  scope  in  the  conclusion 
of  the  rule  fsV  since  6  only  depends  on  modal  variables  in  A  and  is  independent  of  the 
bound  variables  in  T.  In  the  fs.atom  rule  the  instantiation  for  existentially  quantified 
variables  is  obtained  by  unifying  P  with  P'  in  the  context  T.  6  is  a  solution  to  the 
unification  problem  A;  T  b  P  =  P'  where  P  and  P'  are  higher-order  patterns. 

There  is  still  some  non- determinism  left  in  Cg,  which  needs  to  be  resolved  in  an 
actual  implementation.  In  the  us.atom  rule,  we  do  not  specify  which  clause  from  T  we 
pick  and  focus  on.  Moreover,  usually  the  order  of  clauses  in  T  is  determined  by  the  order 
of  the  clauses  in  the  program.  Logic  programming  interpreters  then  usually  try  the 
clauses  in  T  in  order  and  do  not  backtrack  over  different  choices  since  this  is  considered 
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Uniform  Proof: 


A;  T,  x:A,  r'  >  A  M  P/(A',  9) 
- us_atom 

A;r,4r'Ap/(A'«) 

A;  T,  x  :  Ai  -A-  A2/ (A,  9) 

- us  — A 

A;T  AUA1-+A2/(A,e) 


Focused  Proof 


A;  T  h  P'  =  P/( A',9) 

- - fs_atom 

A;T  >  P1  M  P/( A',  6) 


A-,T,x:A1^A2/(A,0) 

- usV 

A;r-%IIs:  A!.A2/(A,0) 


A,  rr::(ri-^4i);  T  S>  [idr,  u[\dr\/x\A2  — >  P j (A7,  (9,  U/u ))  a  is  a  new  modal  variable 
- f sV 

A;r  >  ILx  :  Ai.A2  M  P/{A',6) 


A;r>  a 


i  M  P/( Ai,  0i)  Ai;  [6>r]r  191}A2/(A2, 02) 

- f  s  -4 

A;  r  »  A2  -  Ai  M  P/( A2,  \e2\0f) 

Figure  4.4:  Uniform  deduction  system  for  Cg  with  substitutions 


too  expensive.  This  choice  renders  the  search  strategy  incomplete  in  practice.  However, 
the  presented  deductive  system  Cg,  which  generates  answer  substitutions,  is  sound  and 
complete,  as  expressed  by  the  following  two  theorems. 


Theorem  62  (Soundness) 


1.  If  A;  T  A/(A',9)  then  for  any  modal  substitution  ■  b  p  :  A'  we  have 

ir  IIpMa 


j 

2.  If  A;  T  S>  A  — >  Pf  (A',  9)  then  for  any  modal  substitution  ■  h  p  :  A'  we  have 

r  »  [W«p  -0  \\p\g[p. 


Proof:  Structural  simultaneous  induction  on  the  derivation  T>. 
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Case  V 


A;r,a;:A1^A2/(A,0) 

- usV 

A;  T  -A->  Ux  :  A1.A2/(A,9) 


[[pl^l(r>  X'-Ai)  [[pj0p2 
IIpMn^IIpM^i)  I[pFp2 

[[p]0](r)^nx:[[p]0](^).[[p]0]^ 

Mom 

A;r,c:  V(A,0) 

Lse  - us 

AirAj^A./IA,#) 

[[#](r,c:  A)  —  [[p]9]42 
[HsKr),c  :  IIpMA)  -A  I[p]#]42 
[M«](r)  [H#1(A)  -  HpMA 

l[pM(r)  —  [[p]0](A  -*  A) 


by  i.h. 

by  substitution  definition 
by  rule 

by  substitution  definition 


by  i.h. 

by  substitution  definition 
by  rule 

by  substitution  definition 


A;  r  »  A1  M  p/{ a1;  0 o  ai;  [0i]r  [PilA2/(A2,  e2) 

Case  V  = - - - fs  — » 

A]r^A2^AlMp/(A2,{d2j91) 

[[p]#ilr  >  [[p]Pipi  [[p]0i]P  by  i.h. 

let  p  =  \p'\e2 

mmoiW  ^  mm°M2  byi.h. 

[[pl02][^i]r  »  [[p'l^M^iPi  ->  mh][ei]A2  [[p'l^M^il-P  by  rule 

[[p']P2]  [0ijr  »  -A-  A2)  — P  by  substitution  definition 


A,  u::(rhA);  T  >  [idr,  u[idr]/a;]A2  M  P/{ A',  (9,  U/u )) 

Case  V  = - fsV 

A;  r  »  Ux  :  AX.A2  P/( A',  0) 

u  is  a  new  modal  variable  by  assumption 

llp}^,U/u)]T  >  |[p](P,  U / u)]([idr,  u[idr]/a;]A2)  [|p](P,  f//u)]P  by  i.h. 

•  b  p  :  A' 
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[M(0,t//«)]r  =  [HS]r 
M(o,u/u)\p=Me\P 

l[p\(8,uiu)\  =  [M», 

[M(«)]r  »  ([id[M9|r,  \p\Vlx\At) 

A'  h  (»,U/u)  :  (A,u-ri-4i) 

A'  h  $  :  A 

A';  [9]r  h  U  :  [fl]^ 

•;W[0]ri-Hc/: 

•;  [[p]9]r  h  \p\U  :  IW91-4! 

■;[M9]r»nI:[Me]41.[we]42 

■;IM9]r»[W«](n^:41.42)^ 


by  strengthening 
by  strengthening 


[[p](0)]P  by  substitution  dehnition 

by  invariant 
by  inversion 
by  inversion 

by  substitution  property  (lemma  41) 
by  composition  lemma  42 
[[pJdjP  by  rule 

9\P  by  substitution  dehnition 


A;rhP'  =  P/(A',9) 

Case  V  = - fs.atom 

A;  T  >  P'  M  P/( A',  6) 

VOW  =  Mp 

A'  h  9  :  A 

for  any  modal  substitution  p  such  that  •  ( 

MMP'  =  HM-P 
IW»]P'  =  WpV\p 
r  »  Hp]»]p'  -U  [[rM»]P 


by  soundness  of  pattern  unification  60 

by  lemma  59 

p  :  A' 

by  substitution  property  (lemma  41) 
by  substitution  property  (lemma  41) 

by  rule 

□ 


Theorem  63  (Completeness) 

1.  If  •;  [p]r  — —>  \p\A  for  a  modal  substitution  p,  s.t.  ■  b  p  :  A 

then  A;  T  A/ (A',  9)  for  some  9  and  p  =  [ p'}9  for  some  p'  s.t.  ■  b  p'  :  A'. 

2.  If  •;  [p]r  \p\A  — — >  lp}P  for  a  modal  substitution  p  s.t.  ■  b  p  :  A 

then  A;r>4  — ^  P/  (A',  0)  for  some  9  and  p  =  [p']d  for  some  p' ,  s.t.  -bp':  A'. 

Proof:  Simultaneous  structural  induction  on  the  first  derivation. 
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Case  V  =  T  Ux  :  Ax.A2 

[p]r  M(ILr  :  Ai.A2) 

Mr  Ux  :  [p]Ai.[p]i2 

Mr, x :  IpIai  M A2 

M(r^ :  ^i)  M^2 

A;  r,  x  :  A 1  A2/ (A',  0)  and  p  =  \p'\0 

A-T  -A^Hx:  A1.A2/(A'16) 

Case  X>  =  T  Ai  — >  A2 

Mr  M(A  ->  40 

Mr  MMO  -»•  M(4) 

Mr,c:[p]4  [pp2 

M(r,c:A)  [pp2 

A;  r,  c:A1  AA  A2/ (A',  0')  and  p  =  {p'\0 
A;  T  —>  Ai  — >  t42/ (A',  O') 


by  assumption 
by  substitution  definition 
by  inversion 
by  substitution  definition 
by  i.h. 
by  rule 


by  assumption 
by  substitution  definition 
by  inversion 
by  substitution  definition 
by  i.h. 
by  rule 


Case  V  =  T  >  A2  — >  Ai  -M  P 

Mr  »  M(A  4,)  M  \p\p 
Mr  »  M(42)  ->  M(A)  \p\p 
Mr  »  M-4,  -U  [p\p 
Mr  ~  MM 

A;T  >  Ai  -A  P/(A1,01)  and  p  -  [p']0i 

IM#i]r  -A  [H#i]42 

H([«jr)  -A  MKiftp,) 

A,;  |»i]r  -M  Wdj/IA,,  d2)  and  pf  =  [/](%) 
A;  r  »  (42  -*  4t)  M  P/( A2,  l$2]0t) 
p  =  M'lMFi 


by  assumption 
by  substitution  definition 
by  inversion 
by  inversion 
by  i.h. 

by  composition  lemma  42 
by  i.h. 
by  rule 


Case  D  =  T>  Ux:Ai.A2  — ^  P 

•;  Mr  »  MXixMM)  -A  [P]p 
•;  Mr  »  nn:M4i.M42  -A  [ftp 
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by  assumption 
by  substitution  definition 
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•;  Mr  >  [idMr,f//i](MA2) 
•;  Mr  h  U  :  [pp, 

•  b  p  :  A 

•  b  (p,u/u)  :  (A,  UV.Y'rAi 


IP 


for  some  U 


by  inversion 
by  inversion 
by  assumption 
by  rule 


[idjpjr,  U/x\ \p\A2  =  [p]([idr,  U/x]A2)  =  [p,  P/u]([idr,  «[idr]/z]A2)  where  u  is  new 

by  substitution  property 

•;  {p,U/ujT  3>  [p,  C//u]([idr,u[idr]/x]p42)  — ^  [p,  U/ujP  by  substitution  definition 
A,  uv.Y'rAi]  T  S>  [idr,  u[idr]/a:]A2  — ^  P/(A',  d')  and 

(p,U/u)  =  \p'\0'  for  some  O'  and  p'  by  i.h. 

let  O'  =  ( 0,U'/u ). 

(p,U/u)  =  lp'l(0,U'/u)  =  ([p'161,  IpjU'/u)  by  substitution  definition 


p  =  Ip' jO  and  U  =  [ ojU ' 

by  syntactic  equaltiy 

A;  r  >  r lx:A1.A2  AU  P/( A',  0) 

by  rule 

Case  V  =  T  >  P  -A+  p 

s  Mr  »  [pF  lp]P 

by  assumption 

\p}P  =  \P]P  and  •  b  p  :  A 

A;  T  b  P  =  P  /  (A',  0)  and  there  exists  a  p'  s.t.  -bp':  A'  and  p  =  \p'\0 

by  completeness  of  higher-order  pattern  unification  (lemma  61) 
A;  T  >  P  -U  p  /  (A',  0)  by  rule 

□ 


In  the  next  section,  we  extend  this  system  Cq  to  include  memoization. 

4.5.3  Tabled  uniform  proofs 

The  idea  behind  tabled  uniform  proofs  is  to  extend  our  two  basic  judgments  with  a 
table  T  in  which  we  record  atomic  sub-goals  and  the  corresponding  answer  substitutions 
and  proof  terms.  A  subgoal  is  a  sequent  A;  T  P  where  A  describes  the  existential 
variables,  T  is  a  program  context  and  P  is  an  atomic  goal,  which  we  need  to  derive 
from  T.  When  we  discover  the  sub-goal  A;  Y  —A  P  for  the  first  time,  we  memoize 
this  goal  in  the  table.  Note  that  the  sequent  A;  T  —A  P  might  potentially  contain 
existential  variables.  Once  we  have  proven  the  sub-goal  A;  T  —A  P,  we  add  the  answer 
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substitution  A'  b  9  :  A  to  the  table.  We  keep  in  mind  that  we  are  silently  generating 
proof  terms  together  with  answer  substitutions.  We  assume  that  some  predicates  are 
designated  as  tabled  predicates  where  we  record  subgoals  and  corresponding  answers. 
For  predicates  not  designated  as  tabled  the  us.atom  rule  still  applies. 

Definition  64  (Table)  A  table  entry  consists  of  two  parts:  a  goal  A;  T  — P  and 
a  list  A  of  pairs,  answer  substitutions  A'  b  9  :  A  and  proof  terms  U ,  such  that 
A';  [0]r  — A  \9\U  :  [ 6\P  is  a  solution.  A  table  T  is  a  collection  of  table  entries. 

The  table  is  a  store  of  proven  and  still  open  conjectures.  The  open  conjectures  are 
the  table  entries  that  have  an  empty  list  of  answers.  The  proven  conjectures  (lemmas) 
are  the  table  entries  that  have  a  list  of  answer  substitutions  associated  with  them.  As 
proof  terms  are  generated  and  stored  together  with  answer  substitutions,  we  also  have 
the  actual  proof  for  the  given  conjecture.  We  will  design  the  inference  rules  in  such  a 
way  that  for  any  solution  in  the  table  A';  [0]T  — \6\U  :  \0\P  there  exists  a  derivation 
A;  T  -d A  U  P/(A',  6).  We  will  keep  all  the  previous  inference  rules,  but  keep  in  mind 
that  we  are  silently  passing  around  a  table  T .  Any  substitution  we  apply  to  T  and 
P  (see  for  example  the  fs  — »  rule)  will  not  effect  the  table.  This  is  important  because 
we  do  want  to  have  explicit  control  over  the  table.  The  application  of  inference  rules 
should  not  have  any  undesired  effects  on  the  table.  The  main  judgments  are 

T;  A;  T  U  :  A/(A',  9,T') 

T;A;T»A  -U  S  :  P/(A',  9,  V) 

In  addition  to  the  us.atom  inference  rule,  we  will  have  the  rules  extend  and  retrieve. 
The  extend  rule  adds  a  subgoal  and  its  answer  to  the  table.  When  we  encounter  a  new 
subgoal,  we  add  a  new  entry  with  an  empty  answer  list  to  the  table.  Once  we  have 
proven  this  subgoal,  we  add  the  answer  substitution  and  proof  term  to  its  answer  list, 
and  we  can  later  use  it  as  a  lemma,  retrieve  allows  us  to  close  a  branch  by  applying 
a  lemma  from  the  table.  If  we  are  proving  A;  T  — where  T  and  P  may  contain 
existential  variables  and  we  have  a  proof  for  A';  [0]T  — \9\U  :  [0]P  in  the  table  then 
we  can  just  re-use  it  by  applying  the  answer  substitution  9  and  substituting  the  proof 
term  \6\U  for  it.  Applying  the  retrieve  rule  corresponds  to  introducing  an  analytic  cut 
in  the  proof  using  a  lemma  from  the  table. 
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Before  we  present  these  two  rules,  we  define  when  a  goal  is  a- variant  to  a  table 
entries.  We  consider  A;  Y  —A  P  a  variant  of  A';  P  —A  P'  if  there  exists  a  renaming 
of  the  ordinary  variables  in  Y  and  the  modal  variables  in  A  such  that  A;  Y  —A  P  is 
equal  to  A';  P  -A  P' . 

Definition  65  (Variant)  The  goal  A;  Y  —A  P  is  a  variant  of  A';  A  —A  P'  if 

•  there  exists  a  bijection  p  between  the  modal  variables  in  A  and  A'  such  that 
A'  bp  :  A. 

•  there  exists  a  bijection  a  between  the  ordinary  variables  in  [pjT  and  P  such  that 

A';  r  h  a  :  [p]r 

•  and  [a][p]P  =  P' . 

This  definition  of  variant  can  be  extended  to  terms.  We  say  a  term  A;  Y  b  U  is  a 
variant  of  term  A';  P  b  U'  if  there  exists  a  bijection  between  the  modal  variables  in 
A  and  A'  and  a  bijection  between  the  ordinary  variables  in  Y  and  P.  Next,  we  define 
when  two  modal  substitutions  are  variants  of  each  other. 

Definition  66  The  m.odal  substitution  Aj  b  A  :  Ax  is  a  variant  of  A2  b  d2  :  A2 

•  Ai  b  •  :  •  is  a  variant  of  A2  b  •  : 

•  A\  b  ( 9i,U/u )  :  (Ai,  m::(Ti-A))  is  a  variant  of  A2  b  (62,N/u)  :  (A2, 
iff 

1.  A[  b  9i  :  Ax  is  a  variant  of  A2  b  02  :  A2 

2.  Aj^;  T  b  U  is  a  variant  of  A2;  b  N 

Now  we  can  define  the  three  main  operations  on  the  table,  extending  the  table, 
inserting  an  answer  in  the  table  and  retrieving  an  answer  from  the  table. 

Definition  67  (extend)  extend(T ,  A;  Y  —A  P)  =  T 

Let  T  be  a  table,  A;  Y  —A  p  be  a  goal. 

•  If  there  exists  a  table  entry  (A';  T'  —A  Pf  A)  in  T  and  A  is  non-empty  such  that 
A';  T'  -A  P'  is  a  variant  of  A;  Y  -A  P  then  return  T. 
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•  If  there  exists  no  table  entry  (A';  D  — A  P' ,  A)  in  T  such  that  A';  D  -A  p’  is  a 
variant  of  A]  T  — A  P,  then  we  obtain  the  extended  table  T  by  adding  A;  T  — A  P 
to  the  table  T  with  an  empty  solution  list. 

Definition  68  (insert)  insertfT,  A;  T  -A  P,  U,  6)  =  T' 

Let  T  be  a  table,  A;  T  — "A  P  be  a  goal  and  A'  b  6  :  A  be  a  corresponding  answer 
substitution  and  U  a  proof  term  such  that  A;T  b  U  :  P.  Let  (A^Tj  — A  Pi,  A)  be  in 
the  table  T  and  T,;  -A  Pt  is  a  variant  of  A;  T  -A  P.  If  there  exists  no  A'  h  6**  :  A* 
in  the  answer  substitution  list  A,  such  6i  is  a  variant  of  6,  then  we  add  A'  b  6  :  A 
together  with  the  proof  term  U  to  the  the  answer  list  A. 

If  we  generate  and  try  to  insert  an  answer  substitution  9  which  already  exists  in 
the  answer  list,  then  we  fail. 

Definition  69  (retrieve)  retrieve(T ,  A;  V  —A  P)  =  (A',  6,  U) 

Let  T  be  a  table  and  A;  T  -A  P  be  a  goal.  If  there  exists  a  table  entry  (A^T*  -A 
Pi,Ai)  such  that  A^T*  -A  Pi  is  variant  of  A;  T  -A  P  and  there  exists  an  answer 
substitution  A'  b  d,  :  A  together  with  a  proof  term  U  in  Aj  then  return  (A',  d*,  U). 

Now  we  can  give  the  additional  rules  extend  and  retrieve. 

extend (T,  A;  (I\  x:A,  V)  -A  P)  =  %_ 

T\,  A;  (r,  x:A,  D)  >  A  — ^  S  :  P/( A',  9,  T2) 

insert(T2,  A;  (r,  x:A,  D)  -A  P,  U,  A',  9)  =  T3 

- - - extend 

T;  A;  (r,  x:A,  D)  -A  x  ■  S  :  P/{  A',  9,  %) 

retrieved;  A;  Y  -A  P)  =  (A',  9,  U ) 
- retrieve 

T;  A;  T  -A  U  :  P/(A',9,T) 

The  rule  extend  is  applicable,  even  if  a  table  entry  already  exists.  In  this  case,  the 
goal  A;  T  -A  P  will  not  be  added  to  the  table,  but  we  will  still  try  to  find  a  solution 
for  the  modal  variables  in  A.  This  is  correct,  since  we  may  need  to  generate  more  than 
one  answer  substitution  9  which  needs  to  be  added  to  the  the  table. 

If  we  discover  a  sub-goal  P  in  a  context  T,  which  is  already  in  the  table  T  but  with 
an  empty  answer  substitution  list,  then  we  have  discovered  a  loop  in  the  computation. 
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No  inference  rule  is  applicable,  and  therefore  computation  fails.  The  extend  rule  is  not 
applicable,  since  we  extend  will  only  succeed  if  the  subgoal  P  in  the  context  T  is  not 
already  in  the  table  or  it  is  in  the  table,  but  it  must  have  a  non-empty  answerlist.  The 
retrieve  rule  is  also  not  applicable,  since  we  have  an  empty  answer  list.  The  definitions 
of  insert  prevents  us  from  inferring  the  same  solution  twice.  If  a  sub-goal  P  is  already 
in  the  table,  but  has  some  answers  in  the  answer  list  A,  then  we  retrieve  the  answers. 
As  we  might  need  additional  answers  for  P  which  are  not  already  in  the  table  yet,  we 
need  to  still  be  able  to  apply  extend  rule. 

The  presented  inference  rules  leave  several  choices  undetermined.  For  example,  we 
do  not  specify  the  order  in  which  we  use  program  clauses.  This  choice  was  already 
present  in  the  non-tabled  system.  Similarly,  the  rules  to  allow  memoization  leave 
open  in  what  order  we  retrieve  answers,  when  to  retrieve  answers  and  when  to  apply 
the  extend  rule.  In  an  implementation,  we  need  to  resolve  these  choices.  We  have 
chosen  a  high-level  declarative  description  of  the  memoization  based  proof  search.  An 
advantage  is  that  is  is  abstract  and  clearly  illustrates  the  use  of  memoization  as  analytic 
cut.  On  the  other  hand,  failure  in  such  a  proof  system  is  implicit,  i.e.  if  no  inference 
rule  is  applicable,  then  we  fail,  “backtrack”,  and  try  another  rule.  Upon  failure,  we 
backtrack  and  reset  the  state  of  the  table.  In  an  implementation,  the  table  T  is 
usually  implemented  via  a  global  variable,  and  persists,  even  if  proof  search  fails. 
Unfortunately,  there  is  no  way  to  express  the  persistence  of  the  table  in  the  proof 
system  without  making  failure  explicit  in  the  deductive  system. This  is  beyond  the 
scope  of  this  thesis.  We  are  still  able  to  prove  soundness  of  memoization-based  search, 
however  completeness  and  termination  are  harder  to  establish. 

Theorem  70  (Soundness) 

1.  If  V  :  T;  A;  T  A/{ A',  9,  T)  then  we  have  £  :  A;  T  A/{ A',  9) . 

2.  If  V  :  T;  A;  T  »  A  -U  P/(A',  9,  T)  then  we  have  £  :  A;  T  >  A  M  P/{ A',  9) . 

Proof:  Structural  simultaneous  induction  on  the  derivation  V.  The  proof  is  straight¬ 
forward  using  the  fact  that  for  every  answer  in  the  table  we  have  a  proof  term,  this 
proof  term  can  be  expanded  into  a  proof.  □ 
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As  we  mentioned  earlier,  there  are  several  choices  left  undetermined  in  the  given 
proof  system  which  we  need  to  resolve  in  an  actual  implementation.  The  multi-stage 
strategy  discussed  earlier  is  one  possible  solution.  In  this  strategy  we  proceed  in  lock- 
steps.  First,  we  apply  the  extend  rule  until  all  clauses  from  T  have  been  tried,  and  then 
allow  the  application  of  the  retrieve  rule.  The  strategy  also  restricts  the  retrieve  rule, 
i.e.  only  answers  from  previous  stages  can  be  retrieved.  Alternatively,  we  could  use 
SCC  scheduling  (strongly  connected  components),  which  allows  us  to  consume  answers 
as  soon  as  they  are  available  [62],  As  mentioned  earlier,  the  proof  system  does  not 
give  us  an  explicit  notion  of  failure  and  the  table  is  not  persistent  across  branches. 
In  a  real  implementation,  we  make  the  table  globally  available.  Moreover,  we  do  not 
want  to  retract  all  our  steps,  when  we  discover  a  loop.  Instead,  we  want  to  freeze 
the  current  proof  state,  suspend  solving  the  goal  and  later  resume  computation.  After 
some  answers  have  been  generated  for  the  sequent  T  AA  P,  we  awaken  the  suspended 
goal  and  resume  computation  of  the  pending  sub-goals.  Finally,  we  want  to  point  out 
that  although  we  used  variant  checking  in  the  definitions,  it  is  possible  to  extend  and 
modify  them  to  allow  subsumption  checking. 

4.5.4  Related  work  and  conclusion 

This  proof-theoretic  view  on  computation  based  on  memoization  provides  a  high-level 
description  of  a  tabled  logic  programming  interpreter  and  separates  logical  issues  from 
procedural  ones  leaving  maximum  freedom  to  choose  particular  control  mechanisms. 
In  fact,  it  is  very  close  to  our  prototype  implementation  for  Elf.  So  far  all  descriptions 
of  tabling  are  highly  procedural,  either  designed  as  an  extension  of  SLD  resolution  [65] 
or  to  the  WAM  abstract  machine [62].  Certificates,  which  provide  evidence  for  the  ex¬ 
istence  of  a  proof,  have  been  added  to  tabled  logic  programming  by  Roychoudhury  [61] 
and  are  called  justihers.  The  relationship  between  the  certificate  and  SLD  resolution 
is  extrinsic  rather  then  intrinsic  and  needs  to  be  established  separately.  The  proof- 
theoretical  characterization  offers  a  uniform  framework  for  describing  and  reasoning 
about  program  clauses,  goals  and  certificates  (proof  terms).  It  seems  possible  to  ap¬ 
ply  the  techniques  described  to  other  logic  programming  languages  such  as  AProlog. 
Linear  logic  programming  [30,  8]  has  been  proposed  as  an  extension  of  higher-order 
logic  programming  to  model  imperative  state  changes  in  a  declarative  (logical)  way. 
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We  believe  our  techniques  can  be  extended  to  cover  this  case,  but  it  requires  some  new 
considerations.  In  particular,  we  plan  to  investigate  the  interaction  between  resource 
management  strategies  [10]  or  constraints  [27]  with  tabling. 

With  tabled  uniform  proof  search  we  will  find  fewer  proofs  than  with  uniform  proofs. 
For  example  in  the  subtyping  example  given  in  Sec.  4.1  the  query  sub  zero  zero  has 
infinitely  many  proofs  under  the  traditional  logic  programming  interpretation  while  we 
find  only  one  proof  under  the  tabled  logic  programming  interpretation.  However,  we 
often  do  not  want  and  need  to  distinguish  between  different  proofs  for  a  formula  A, 
but  only  care  about  the  existence  of  a  proof  for  A  together  with  a  proof  term.  In  [50] 
Pfenning  develops  a  dependent  type  theory  for  proof  irrelevance  and  discusses  potential 
applications  in  the  logical  framework.  This  allows  us  to  treat  all  proofs  for  A  as  equal 
if  they  produce  the  same  answer  substitution.  In  this  setting,  it  seems  possible  to  show 
that  search  based  on  tabled  uniform  proofs  is  also  non-deterministically  complete,  i.e. 
if  computation  fails,  then  there  exists  no  proof. 


4.6  Case  studies 

In  this  section,  we  discuss  experiments  with  tabled  higher-order  logic  programming. 
We  compare  the  performance  of  proof  search  based  memoization,  depth-first  search, 
iterative  deepening  using  two  applications:  1)  bi-directional  type  checking  using  sub¬ 
typing  and  intersection  types  and  2)  parsing  into  higher-order  abstract  syntax.  As  the 
experiments  demonstrate  proof  search  based  on  memoization  can  lead  to  substantial 
performance  improvements,  making  the  execution  of  some  queries  feasible. 

While  computation  using  memoization  yields  better  performance  for  programs  with 
transitive  closure  or  left-recursion,  Prolog-stylc  evaluation  is  more  efficient  for  right 
recursion.  For  example,  Prolog  has  linear  complexity  for  a  simple  right  recursive  gram¬ 
mar,  but  with  memoization  the  evaluation  could  be  quadratic  as  calls  need  to  be 
recorded  in  the  tables  using  explicit  copying.  Therefore  it  is  important  to  allow  tabled 
and  non-tabled  predicates  to  be  freely  intermixed  and  be  able  to  choose  the  strategy 
that  is  most  efficient  for  the  situation  at  hand.  Hence  in  the  current  prototype,  the 
user  declares  predicates  to  be  tabled,  if  he/she  wishes  to  use  memoization  for  it.  Mix¬ 
ing  tabled  and  non-tabled  predicates  is  essential,  in  order  to  obtain  an  efficient  proof 
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search  engine. 

4.6.1  Bidirectional  typing  (depth-first  vs  memoization) 

In  this  section,  we  discuss  experiments  with  a  bi-directional  type-checking  algorithm 
for  a  small  functional  language  with  intersection  types  which  has  been  developed  by 
Davies  and  Pfenning  [17].  Type  inference  in  a  functional  language  with  subtyping  and 
intersection  types  is  usually  considered  impractical,  as  no  principal  types  exist.  The 
idea  behind  bi-directional  type-checking  is  to  distinguish  expressions  for  which  a  type 
can  be  synthesized  from  expressions  which  can  be  checked  against  a  given  type.  The 
programmer  specifies  some  types  to  guide  inferring  a  type  for  certain  expressions. 

Inferable  /  ::=  x  \  e  \  I  0  |  /  1  |  app  IC  \  C  :  A 

Checkable  C  ::  =  I  \  lam  x.C  |  let  u  —  I  in  C  \ 

case  I  of  e  =>  C\\x  0  C2\x  1  C3 

The  intention  is  that  given  a  context  T  and  an  expression  /,  we  use  type-inference 
to  show  expression  /  has  type  A  and  type-checking  for  verifying  that  expression  C  has 
type  A.  In  an  implementation  of  the  bi-directional  type  checking  algorithm,  there  may 
be  many  ways  to  derive  that  /  has  a  type  A  and  similarly  there  are  more  than  one  way 
to  check  that  C  has  a  given  type  A.  To  discuss  the  full  bi-directional  type-checking 
algorithm  is  beyond  the  scope  of  this  paper  and  the  interested  reader  is  referred  to  the 
original  paper  by  Davies  and  Pfenning  [17]. 

We  use  an  implementation  of  the  bi-directional  type-checker  in  Elf  by  Pfenning. 
The  type-checker  is  executable  with  the  original  logic  programming  interpreter,  which 
performs  a  depth-first  search.  However,  redundant  computation  may  severely  hamper 
its  performance  as  there  are  several  derivations  for  proving  that  a  program  has  a 
specified  type.  For  example,  there  are  approximately  20,000  ways  to  show  that  the 
program  plus  has  the  intersection  type  (nat  — >  nat  — >  nat)  A  (nat  — >  pos  — >  pos)  A 
(pos  — >  nat  — >  pos)  A  (pos  — >  pos  — >  pos).  It  might  be  argued  that  we  are  only 
interested  in  one  proof  for  showing  that  plus  has  the  specified  type  and  not  in  all  of  them. 
However,  it  indicates  that  already  fairly  small  programs  involve  a  lot  of  redundant 
computation.  More  importantly  than  succeeding  quickly  may  be  to  fail  quickly,  if 
plus  has  not  the  specified  type.  Failing  quickly  is  essential  in  debugging  programs 
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and  response  times  of  several  minutes  are  unacceptable.  In  the  experiments,  we  are 
measuring  the  time  it  takes  to  explore  the  whole  proof  tree.  This  gives  us  an  indication 
how  much  redundancy  is  in  the  search  space.  When  checking  a  term  C  against  a  type  A, 
we  use  memoization.  The  proof  search  based  on  memoization  uses  strengthening  and 
variant  checking.  Using  the  refinement  type-checker,  we  check  programs  for  addition, 
subtraction,  and  multiplication.  Note  that  in  order  to  for  type-check,  for  example,  the 
multiplication  program,  we  need  to  also  type-check  any  auxiliary  programs  used  such 
as  addition  and  shift.  Table  4.1  summarizes  the  results  for  hireling  all  solutions  to  a 
type-checking  problem.  The  number  associated  with  each  program  name  denotes  the 
depth  of  the  intersection  type  associated  with  it.  For  example,  plus4  means  we  assigned 
4  types  by  using  intersections  to  plus,  nrultla  indicates  we  associated  one  possible  type 
to  the  multiplication  program.  If  a  program  label  is  marked  with  (np),  it  means  this 
query  does  not  have  a  solution  and  the  type-checker  should  reject  the  query. 


Program 

Depth- First 

Memoization 

^Entries 

^SuspGoals 

plus ’4 

483.070  sec 

2.330  sec 

151 

48 

plus4 

696.730  sec 

3.150  sec 

171 

74 

plus4(np) 

22.770  sec 

1.95  sec 

143 

56 

sub ’la 

0.070  sec 

0.240  sec 

58 

11 

sub’ 3a 

0.130  sec 

0.490  sec 

92 

20 

sub  la 

3.52  sec 

7.430  sec 

251 

135 

sub  lb 

3.88  sec 

7.560  sec 

252 

138 

sub3a 

10.950  sec 

9.970  sec 

277 

167 

sub3b 

10.440  sec 

11.200  sec 

278 

170 

multi  (np) 

1133.490  sec 

4.690  sec 

217 

83 

mult  la 

807.730  sec 

4.730  sec 

211 

78 

mult  lb 

2315.690  sec 

6.050  sec 

226 

101 

multlc 

2963.370  sec 

5.310  sec 

226 

107 

mult4 

oo 

17.900  sec 

298 

270 

mult4(np) 

oo 

13.140  sec 

275 

194 

Table  4.1:  Finding  all  solutions:  depth-first  vs  memoization-based  search 
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For  type-checking  programs  plus  and  multiplication  proofs  search  based  on  memo- 
ization  outperforms  depth-first  search.  This  is  surprising,  as  no  sophisticated  indexing 
is  used  for  accessing  the  table.  It  indicates  that  already  simple  memoization  mechanism 
can  substantially  improve  performance.  In  the  case  of  multiplication,  it  makes  type¬ 
checking  possible.  We  stopped  the  depth-first  search  procedure  after  lOh.  Of  course 
proof  search  based  on  memoization  has  some  overhead  in  storing  and  accessing  goals 
in  the  table.  As  indicated  with  subtraction  programs,  this  overhead  might  degrade 
the  performance  of  memoization  based  proof  search.  When  type-checking  subtraction 
program  depth-first  search  performs  better  than  memoization-based  search  for  the  first 
5  sample  programs.  In  the  last  example  sub3b  however  memoization-based  search  wins 
over  depth-first  search. 


Program 

Depth- First 

Memoization 

^Entries 

#SuspGoals 

plus ’4 

0.08  sec 

0.180  sec 

54 

0 

plus4 

0.1  sec 

0.430  sec 

72 

0 

plus4(np) 

22.770  sec 

1.95  sec 

143 

56 

sub ’la 

0.050  sec 

0.240  sec 

64 

11 

sub’ 3a 

0.110  sec 

0.410  sec 

92 

20 

sub  la 

0.250  sec 

6.210  sec 

251 

135 

sub  lb 

0.250  sec 

5.020  sec 

242 

121 

sub3a 

0.280  sec 

7.80  sec 

277 

161 

sub3b 

0.350  sec 

8.160  sec 

278 

164 

multl(np) 

1133.490  sec 

4.690  sec 

217 

83 

mult  la 

0.160  sec 

2.900  sec 

201 

60 

mult  lb 

0.180  sec 

4.090  sec 

222 

90 

multlc 

0.170  sec 

2.930  sec 

211 

60 

mult4 

0.250  sec 

7.150  sec 

272 

181 

mult4(np) 

oo 

13.020  sec 

275 

194 

Table  4.2:  Finding  the  first  solution:  depth-first  vs  memoization-based  search 

We  include  also  a  comparison  between  the  two  search  strategies,  when  we  stop 
after  the  first  answer  has  been  found  (see  table  4.2).  It  is  apparent  that  currently 
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finding  the  first  solution  to  a  solvable  type-checking  problem  (i.e.,  it  is  provable  that 
the  program  has  the  specified  type)  always  takes  longer  with  proof  search  based  on 
memoization  -  in  some  cases  considerably  longer  (see  subtraction  and  multiplication). 
This  is  an  indication  that  accessing  the  table  is  still  quite  expensive  in  the  current 
implementation.  This  comes  as  no  real  surprise,  because  no  sophisticated  techniques 
such  as  term  indexing  are  used.  This  problem  will  be  addressed  in  Chapter  5.  The  other 
reason  for  the  poor  performance  of  memoization-based  search  is  due  the  multi-stage 
search  strategy.  Although  this  strategy  is  relatively  easy  to  implement  and  understand, 
it  restricts  retrieval  of  answers  present  in  the  table  to  answers  generated  in  previous 
stages.  This  causes  subgoals  to  be  suspended,  although  answers  might  be  available, 
and  solving  those  subgoals  is  delayed.  For  this  reason,  XSB  uses  the  SCC  (strongly 
connected  component)  scheduling  strategy,  which  allows  to  re-use  answers  from  the 
table  as  soon  as  they  are  available.  But  the  benefits  of  memoization-based  search  are 
apparent  when  comparing  the  time  it  takes  to  reject  a  program  by  the  type-checker 
as  the  examples  plus4(np)  and  multi (np)  indicate.  Overall,  the  performance  of  the 
memoization  based  search  is  much  more  consistent,  i.e.,  it  takes  approximately  the 
same  time  to  accept  or  reject  a  program. 

To  make  bi-directional  type-checking  reasonably  efficient  in  practice,  Davies  and 
Pfenning  currently  investigate  an  algorithm  which  synthesizes  all  types  of  an  inferable 
term  and  tracks  applicable  ones  through  the  use  of  boolean  constraints.  This  is  however 
far  from  trivial  and  refining  the  Elf  implementation  by  adding  an  extra  argument 
to  the  type-checker  to  memoize  solutions,  complicates  the  type  checker  substantially. 
As  a  consequence,  the  certificates,  which  are  produced  as  a  result  of  the  execution, 
are  larger  and  contain  references  to  the  explicit  memoization  data-structure.  This  is 
especially  undesirable  in  the  context  of  certified  code  where  certificates  are  transmitted 
to  and  checked  by  a  consumer,  as  sending  larger  certificates  takes  up  more  bandwidth 
and  checking  them  takes  more  time.  Moreover,  proving  the  correctness  of  the  type- 
checker  with  special  memoization  support  will  be  hard,  because  we  need  to  reason 
explicitly  about  the  structure  of  memoization.  The  experiments  demonstrate  that 
proof  search  based  on  memoization  has  the  potential  to  turn  the  bi-directional  type¬ 
checking  algorithm  into  a  quite  efficient  type-checker  without  any  extra  effort  on  behalf 
of  the  user. 
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4.6.2  Parsing  (iterative  deepening  vs  memoization) 

Recognition  algorithms  and  parsers  for  grammars  are  an  excellent  way  to  illustrate  the 
benefits  of  tabled  evaluation.  Warren  [68]  notes  that  implementations  of  context-free 
grammars  in  Prolog  result  in  a  recursive  descent  recognizer,  while  tabled  logic  pro¬ 
gramming  turns  the  same  grammar  into  a  variant  of  Early’s  algorithm  (also  known  as 
active  chart  recognition  algorithm)  whose  complexity  is  polynomial.  Moreover,  tabled 
logic  programming  allows  us  to  execute  left  and  right  recursive  grammars  that  would 
otherwise  loop  under  Prolog-style  execution  (e.g.  left  recursive  ones).  We  illustrate 
tabled  computation  with  parsing  of  first-order  formulas  into  higher-order  abstract  syn¬ 
tax.  First-order  formulas  are  defined  as  usual. 

Propositions  A  ::=  atom  P  \  ->A  \  A&lA\AmA\A=>A  \  true  |  false  | 

fora  1 1  x.A  |  exists  x.A  |  (4) 

Terms  are  either  constants  or  variables  or  functions  with  arguments.  Atomic  propo¬ 
sitions  are  either  propositional  constant  or  a  predicate  with  terms  as  arguments.  In 
addition  to  the  given  grammar,  we  impose  the  following  precedence  ordering:  -i  > 
&  >  v  >  =>.  Conjunction  and  disjunction  are  left  associative,  while  implication  is 
right  associative. 

fall:  fq  C  Cforall’  ;  I  ;  F)  F ’  (forall  P)> 

<-  ({x:id}  fq  ( (bvar  I  x)  #  C)  F  F’  (P  x)). 

fex:  fq  C  (’exist’  ;  I  ;  F)  F’  (exist  P)} 

<-  ({x:idj-  fq  ((bvar  I  x)  #  C)  F  F’  (P  x)). 

cq  :  fq  C  F  F’  P 

<-  fi  C  F  F’  P. 

%  implication  —  right  associative 
fimp:  fi  C  F  F’  (PI  =>  P2) 

<-  fo  C  F  (’imp’  ;  FI)  PI 
<-  fi  C  FI  F’  P2 . 
ci:  fi  C  F  F’  P 

<-  fo  C  F  F’  P. 

%  disjunction  —  left  associative 
for:  fo  C  F  F’  (PI  v  P2) 

<-  fo  C  F  (’or’  ;  FI)  PI 
<-  fa  C  FI  F’  P2 . 
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The  parser  takes  a  context  for  the  bound  variables,  a  lists  of  tokens  and  returns  a 
list  of  tokens  and  a  valid  formula  represented  in  higher-order  abstract  syntax  [52],  For 
example,  in  the  predicate  fi  C  F  FJ  P,  C  represents  the  context  for  bound  variables, 
F  denotes  the  input  stream,  F’  is  a  sub-list  of  the  input  stream  s.t.  (FI  ;  F’ )  =  F  and 
FI  is  translated  into  the  formula  P.  The  other  predicates  work  similarly.  The  complete 
implementation  of  the  parser  is  given  in  the  appendix.  Initially  the  context  of  bound 
variables  is  empty,  the  first  list  of  tokens  represents  the  input  stream,  the  second 
list  is  empty  and  P  will  eventually  contain  the  result.  Using  higher-order  abstract 
syntax,  variables  bound  in  constructors  such  as  fora  1 1  and  exists  will  be  bound  with 
A  in  Elf.  The  simplest  way  to  implement  left  and  right  associativity  properties  of 
implications,  conjunction  and  disjunction  is  to  mix  right  and  left  recursive  program 
clauses.  Clauses  for  conjunction  and  disjunction  are  left  recursive,  while  the  program 
clause  for  implication  is  right  recursive. 

Such  an  implementation  of  the  grammar  is  straightforward  mirroring  the  defined 
properties  such  as  left  and  right  associativity  and  precedence  ordering.  However,  the 
execution  of  the  grammar  will  loop  infinitely  when  executed  with  a  traditional  logic 
programming  interpreter.  Hence  we  compare  execution  of  some  sample  programs  using 
proof  search  based  on  memoization  and  with  proof  search  based  on  iterative  deepening, 
as  performed  by  the  current  theorem  prover  Twelf  [64],  Iterative  deepening  search 
requires  the  user  to  provide  a  depth-bound.  It  is  worth  pointing  out  that  iterative 
deepening  search  will  stop  after  it  found  its  first  solution  or  it  hits  a  depth-bound, 
while  search  based  on  memoization  will  stop  after  it  found  a  solution  (and  showed  that 
no  other  solution  exists)  or  proved  no  other  solution  exists.  This  means,  we  cannot 
use  iterative  deepening  search  to  decide  whether  a  given  stream  of  tokens  should  be 
accepted  or  not,  while  the  memoization-based  search  yields  a  decision  procedure. 

The  experiments  indicate  that  proof  search  based  on  memoization  provides  a  more 
efficient  way  to  decide  whether  a  given  stream  of  tokens  belongs  to  the  language  of 
formulas.  In  fact  for  input  streams  whose  length  is  greater  than  50  tokens,  we  stopped 
the  iterative  deepening  procedure  after  several  hours. 

Remark  1:  It  is  worth  noting  that  in  general  the  straightforward  approach  of  adding 
an  extra  argument  to  the  non-terminals  of  the  input  grammar  -  representing  the  por¬ 
tion  of  the  parse  tree  that  each  rule  generates  -  and  naturally  to  also  add  the  necessary 
code  that  constructs  the  parse  tree  can  be  extremely  unsatisfactory  from  a  complex- 
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Length  of  input 

Iter,  deepening 

Memoization 

^Entries 

^SuspGoals 

5 

0.020  sec 

0.010  sec 

15 

11 

20 

1.610  sec 

0.260  sec 

60 

54 

32 

208.010  sec 

2.020  sec 

176 

197 

56 

(X) 

7.980  sec 

371 

439 

107 

oo 

86.320  sec 

929 

1185 

Table  4.3:  Comparison  between  iterative  deepening  and  memoization  based  search 

ity  standpoint.  Polynomial  recognition  algorithms  might  be  turned  into  exponential 
algorithm  since  there  may  be  exponentially  many  parse  trees  for  a  given  input  string. 
However  in  this  example,  there  is  exactly  one  parse  trace  associated  to  each  formula, 
therefore  adding  an  extra  argument  to  the  recognizer  only  adds  a  constant  factor. 

Remark  2:  Instead  of  representing  the  input  string  as  a  list,  we  can  store  it  in 
the  database  as  a  set  of  facts.  We  can  think  of  each  token  in  the  input  stream  being 
numbered  starting  from  1.  Then  we  will  store  the  string  as  a  set  of  facts  of  the  form 
word  1  ,forall)  .  word  2  ’ x’  .  etc.  where  word  i  tok_i  represents  the  ith  token 
of  the  input  stream. 


4.7  Related  work  and  conclusion 

A  number  of  different  frameworks  similar  to  Elf  have  been  proposed  such  as  AProlog  [39, 
24]  or  Isabelle  [45,  46].  While  Elf  is  based  on  the  LF  type  theory,  AProlog  and  Isabelle 
are  based  on  hereditary  Harrop  formulas.  The  traditional  approach  for  supporting 
theorem  proving  in  these  frameworks  is  to  guide  proof  search  using  tactics  and  tacticals. 
Tactics  transform  a  proof  structure  with  some  unproven  leaves  into  another.  Tacticals 
combine  tactics  to  perform  more  complex  steps  in  the  proof.  Tactics  and  tacticals 
are  written  in  ML  or  some  other  strategy  language.  To  reason  efficiently  about  some 
specification,  the  user  implements  specific  tactics  to  guide  the  search.  This  means 
that  tactics  have  to  be  rewritten  for  different  specifications.  Moreover,  the  user  has 
to  understand  how  to  guide  the  prover  to  find  the  proof,  which  often  requires  expert 
knowledge  about  the  systems.  Proving  the  correctness  of  the  tactic  is  itself  a  complex 
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theorem  proving  problem.  The  approach  taken  in  Elf  is  to  endow  the  framework 
with  the  operational  semantics  of  logic  programming  and  design  general  proof  search 
strategies  for  it.  The  user  can  concentrate  on  developing  the  high-level  specification 
rather  than  getting  the  proof  search  to  work.  The  correctness  of  the  implementation  is 
enforced  by  type-checking  alone.  The  preliminary  experiments  demonstrate  that  proof 
search  based  on  memoization  offers  a  powerful  search  engine. 
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Chapter  5 

Higher-order  term  indexing 


Efficient  term  indexing  techniques  have  resulted  in  dramatic  speed  improvements  in 
all  major  first-order  logic  programming  and  theorem  proving  systems  and  have  been 
crucial  to  their  success.  Broadly  speaking,  term  indexing  techniques  facilitate  the  rapid 
retrieval  of  a  set  of  candidate  terms  satisfying  some  property  (e.g.  unifiability,  instance, 
variant  etc.)  from  a  large  collection  of  terms.  In  logic  programming,  for  example,  we 
need  to  select  all  clauses  from  the  program  which  unify  with  the  current  goal.  In  tabled 
logic  programming  we  memoize  intermediate  goals  in  a  table  and  reuse  their  results 
later  in  order  to  eliminate  redundant  and  infinite  computation.  Here  we  need  to  find 
all  entries  in  the  table  such  that  the  current  goal  is  a  variant  or  an  instance  of  a  table 
entry  and  re-use  the  associated  answers.  If  there  is  no  such  table  entry,  we  need  to 
add  the  current  goal  to  the  table.  Since  rapid  retrieval  and  efficient  storage  of  large 
collection  of  terms  plays  a  central  role  in  logic  programming  and  in  proof  search  in 
general,  a  variety  of  indexing  techniques  have  been  proposed  for  first-order  terms  (see 
[60]  for  a  survey).  However,  indexing  techniques  for  higher-order  terms  are  missing 
thereby  severely  hampering  the  performance  of  higher-order  systems  such  as  Twclf 
[53],  AProlog  [39]  or  Isabelle  [45]. 

In  this  chapter,  we  present  a  higher-order  term  indexing  technique  based  on  sub¬ 
stitution  trees.  Substitution  tree  indexing  [25]  is  a  highly  successful  first-order  term 
indexing  strategy  which  allows  the  sharing  of  common  sub-expressions  via  substitu¬ 
tions.  Given  the  following  two  terms: 

pred  (h  (g  a))  (g  b)  a  and  pred  (h  (g  b))  (g  b)  a 
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we  can  compute  the  most  specific  linear  generalization  (mslg)  of  both  terms,  which 
is 


pred  (h  (g  *))  (g  b)  a 

where  *  is  a  placeholder.  Then  we  obtain  the  first  term  by  substituting  a  for  the 
placeholder  and  the  second  term  by  substituting  b  for  it. 

In  this  chapter  we  present  an  indexing  technique  for  terms  of  the  dependency  typed 
lambda-calculus,  on  computing  the  most  specific  linear  generalization  of  two  terms. 
However  in  the  higher-order  setting,  the  most  specific  linear  generalization  of  two  terms 
does  not  exist  in  general.  Second,  retrieving  all  terms,  which  unify  or  match,  needs  to  be 
efficient  -  but  higher-order  unification  is  undecidable  in  general.  As  discovered  by  Miller 
[34],  there  exists  a  decidable  fragment,  called  higher-order  patterns.  For  this  fragment, 
unification  and  computing  the  most  specific  linear  generalization  is  decidable  even  in 
rich  type  theories  with  dependent  types  and  polymorphism  as  shown  by  Pfenning  [48]. 
However,  these  algorithms  may  not  be  efficient  in  practice  [57]  and  hence  it  is  not 
obvious  that  they  are  suitable  for  higher-order  term  indexing  techniques. 

In  this  chapter,  we  present  substitution  tree  indexing  for  higher-order  terms  based 
on  linear  higher-order  patterns  (see  Chapter  3).  Linear  higher-order  patterns  refine  the 
notion  of  higher-order  patterns  further  and  factor  out  any  computationally  expensive 
parts.  As  we  have  shown  in  Chapter  3,  many  terms  encountered  fall  into  this  fragment 
and  linear  higher-order  pattern  unification  performs  well  in  practice.  In  this  chapter, 
we  give  algorithms  for  computing  the  most  specific  linear  generalization  of  two  linear 
higher-order  patterns,  for  inserting  terms  in  the  index  and  for  retrieving  a  set  of  terms 
from  the  index  such  that  the  query  is  an  instance  of  the  term  in  the  index.  This 
indexing  structure  is  implemented  as  part  of  the  Twelf  system  [53]  to  speed-up  the 
execution  of  the  tabled  logic  programming  interpreter  [55].  Experimental  results  show 
substantial  performance  improvements,  between  100%  and  over  800%. 

The  chapter  is  organized  as  follows:  In  Section  5.1,  we  present  the  general  idea  of 
higher-order  substitution  trees.  In  Section  5.2,  we  give  algorithms  for  computing  the 
most  specific  linear  generalization  of  two  terms  and  inserting  terms  into  the  index.  Re¬ 
trieval  is  discussed  in  Section  5.3.  In  Section  5.4,  we  present  some  experimental  results 
comparing  the  tabled  higher-order  logic  programming  interpreter  with  and  without 
indexing. 
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5.1  Higher-order  substitution  trees 

The  general  indexing  problem  can  be  described  as  follows:  Given  a  large  set  S  of 
indexed  terms  and  a  single  term  U  called  the  query  term ,  we  have  to  retrieve  quickly 
each  term  V  G  S  such  that  a  retrieval  condition  R  holds  between  U  and  V.  In 
general  the  retrieval  condition  can  be  unifiability,  instance  or  variant  checking,  or 
finding  generalizations.  The  term  indexing  problem  consists  of  finding  a  data-structure, 
called  the  index ,  which  allows  one  to  perform  efficiently  the  following  operations:  1) 
index  maintenance ,  i.e.  changing  the  index  when  terms  are  inserted  (or  deleted)  from 
the  set  of  indexed  terms.  2)  term  retrieval  i.e.  finding  some  (or  all)  terms  V  E  S  s.t. 
V  and  the  query  term  U  are  in  relation  to  each  other. 

To  illustrate  the  notation,  we  give  a  sample  signature  together  with  a  set  of  terms 
we  want  to  index.  We  define  a  type  family  exp  for  expressions.  In  addition,  we  give 
constants  a,  b,  c  and  constructors  f,  g,  h  for  building  expressions  of  this  small  language, 
exp:  type. 

a:  exp.  h:  exp  ->  exp. 

b:  exp.  g:  exp  ->  exp. 

c:  exp.  f:  (exp  ->  exp)  ->  exp. 

Finally,  we  define  a  predicate  pred  which  takes  in  three  arguments,  where  each 
argument  is  an  expression 


pred:  exp  ->  exp  ->  exp  ->  type. 

Next,  we  give  four  examples  of  the  predicate  pred: 

pred  (h  (g  b))  (g  b)  a  (1) 

pred  (h  (g  b))  (g  b)  b  (2) 

Iffiv::  •  hexp.  pred  (h  (h  b))  (g  b)  (f  Ax.u[-])  (3) 

nDu::a::exphexp.  pred  (h  (h  c))  (g  b)  (f  Ax.u[id])  (4) 

The  first  two  predicates  are  closed,  while  predicate  (3)  and  (4)  contain  existential 
variables  v  and  u  respectively  which  are  bound  by  IIn.  Using  these  examples,  we 
will  briefly  highlight  some  of  the  subtle  issues  concerning  the  interplay  of  bound  and 
existential  variables.  The  third  predicate  (3)  refers  to  the  existential  variable  v.  Note 
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that  v  is  associated  with  the  empty  substitution,  although  it  occurs  in  the  context  of 
a  bound  variable  x.  This  means  that  any  instantiation  we  find  for  v  is  not  allowed 
to  depend  on  the  bound  variable  x.  In  contrast,  predicate  (4)  refers  to  an  existential 
variable  u ,  which  is  associated  with  the  identity  substitution.  This  means  that  any 
instantiation  we  find  for  u  may  depend  on  the  bound  variable  x.  We  come  back  to 
this  point  in  Section  5.1.1.  All  these  terms  share  some  structural  information.  For 
example,  the  third  argument  is  same  in  all  the  terms.  When  storing  these  terms,  we 
would  like  to  store  the  subterm  (g  b)  only  once.  Similarly,  when  looking  up  if  a  query 
U  is  already  in  the  index,  we  only  want  to  check  once  if  the  third  argument  of  U  is 
(g  b).  In  other  words,  an  indexing  data-structure  should  allow  us  to  share  common 
structure  and  common  operation. 

In  this  chapter,  we  will  focus  on  substitution  tree  indexing.  We  start  by  giving 
the  general  idea  higher-order  substitution  tree.  To  build  a  higher-order  substitution 
tree,  we  proceed  in  two  steps:  First,  we  standardize  the  terms  and  convert  terms  into 
linear  higher-order  patterns.  Second,  we  represent  terms  as  a  sequence  of  substitutions, 
which  are  stored  in  a  tree. 

5.1.1  Standardization:  linear  higher-order  patterns 

To  get  the  maximum  structure  sharing  across  different  indexed  terms  it  is  important  to 
use  variables  in  a  consistent  manner.  In  first-order  term  indexing  we  therefore  standard¬ 
ize  the  term  before  inserting  a  term  into  an  index.  In  addition,  first-order  term  indexing 
strategies  often  employ  linearization.  When  converting  the  term  into  standard  form 
every  occurrence  of  an  existential  variable  is  represented  as  a  distinct  standardized  ex¬ 
istential  variable.  Together  with  the  linear  term,  we  then  also  store  variable  definitions, 
which  establish  the  equality  between  these  two  variables.  One  of  the  reasons  for  using 
linearization  is  efficiency.  Nonlinear  terms  may  have  multiple  occurrences  of  the  same 
existential  variable  and  therefore  requiring  to  check  whether  different  substitutions  for 
the  same  variable  are  consistent.  Since  such  consistency  checks  can  be  potentially  ex¬ 
pensive  and  may  lead  to  performance  degradation,  most  first-order  indexing  techniques 
rely  on  a  post-processing  step  to  carry  out  the  consistency  checks. 

To  design  a  higher-order  indexing  technique,  we  will  extend  this  notion  of  lineariza¬ 
tion  and  standardization.  We  will  require  that  terms  are  linear  higher-order  patterns. 
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Higher-order  patterns  [34,  48]  are  terms  where  every  existential  variable  must  be  applied 
to  some  distinct  bound  variables.  Linear  higher-order  patterns  (see  Chapter  3)  impose 
some  further  restrictions  on  the  structure  of  terms:  First,  all  existential  variables  must 
occur  only  once.  This  allows  us  to  delay  any  expensive  consistency  checks.  Second,  all 
existential  variables  must  be  applied  to  all  distinct  bound  variables.  This  eliminates 
any  computationally  expensive  checks  involving  bound  variables.  This  observation  to 
restrict  higher-order  patterns  even  further  to  patterns  where  existential  variables  must 
be  applied  to  all  bound  variables  has  also  been  made  by  Hanus  and  Prehofer  [26]  in 
the  context  of  higher-order  functional  logic  programming.  While  Hanus  and  Prehofer 
syntactically  disallow  terms  which  are  not  fully  applied,  we  translate  any  term  into  a 
linear  higher-order  pattern  together  with  some  variable  definitions.  As  we  have  shown 
in  Chapter  3,  performance  of  unification  is  improved  substantially  by  this  technique. 

To  illustrate,  let  us  consider  the  previous  term  pred  (h  (g  b))  (g  b)  (f  Ax.u[-]). 
The  existential  variable  n[-]  occurs  only  once,  but  is  not  applied  to  all  distinct  bound 
variables,  since  v  is  not  allowed  to  depend  on  the  bound  variable  x.  Therefore  u[-]  is  a 
higher-order  pattern,  but  it  is  not  linear.  We  can  enforce  that  every  existential  variable 
occurs  only  once  and  is  applied  to  all  bound  variables,  by  translating  it  into  a  linear 
higher-order  pattern: 

pred  (h  (g  b))  (g  b)  (f  A x.u[x/x\) 
together  with  a  variable  definition, 

Vaxexp  ,u[x/x]  =  i>[-] 

where  u  is  a  new  existential  variable  which  is  applied  to  the  bound  variable  x.  Similarly 
to  first-order  standardization  and  linearization,  linearization  in  the  higher-order  case 
remains  quite  straightforward  and  can  be  done  by  traversing  the  term  once.  The  main 
difference  is  that  the  postponed  variable  definitions  may  be  more  complex,  as  we  have 
discussed  earlier  in  Chapter  3. 

5.1.2  Example 

In  this  section,  we  show  the  substitution  tree  for  the  set  of  terms  given  earlier.  In 
each  node,  we  store  a  set  of  substitutions,  which  we  write  here  as  U  =  i  where  i  is 
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an  internal  modal  variable  and  i  will  always  be  applied  to  all  bound  variables  it  may 
depend  on.  Therefore  it  will  always  be  associated  with  the  identity  substitution  id. 


pred  (h  il  id  )  (g  b)  i2id  =  iO 


(g  b)  =  il 

(h  i3id  HI 

i2id  =  i2 

(f  lax  x.  E  x)=  i2 

a  =  i2 

b  —  i2 

b  =  i3 

c  =  i3 

true 

true 

El  unit 

true 

(1) 

(2) 

(3) 

(4) 

Note  that  variable  definitions  are  stored  at  the  leafs.  By  composing  the  substitutions 
in  the  right-most  branch  for  example  we  get 

[c/*3][f  Ax.  u[id]/i2,  (h  *3[id])/*i][pred  (h  H[id])  (g  b)  i2[id]/i0]io[id] 

which  represents  the  term  (4)  pred  (h  (h  c))  (g  b)  (f  Ax.  w[id]).  Note  that  in 
the  implementation  we  can  omit  the  identity  substitution  «2 [id]/«2  and  maintain  as  an 
invariant  that  the  substitutions  in  each  node  can  be  extended  appropriately. 

Higher-order  substitution  trees  are  designed  for  linear  higher-order  patterns.  As 
discussed  in  Chapter  3,  linearization  of  a  term  may  lead  to  a  linear  term  which  is  not 
necessarily  dependently  typed.  However,  the  term  is  approximately  well-typed  when 
all  the  dependencies  among  types  are  erased.  In  Chapter  2,  we  have  used  erasure  of 
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type  dependencies  to  describe  a  definitional  equality  algorithm.  Here,  we  apply  this 
idea  of  approximate  types  to  define  higher-order  substitution  trees.  The  algorithms 
for  insertion  and  retrieval  in  substitution  trees  are  based  on  rnslg  and  unifiability,  but 
types  itself  do  not  play  a  role  when  computing  the  rnslg  and  unifiers.  We  know  the  term 
is  well-typed  before  it  is  inserted  into  the  substitution  tree,  and  it  will  be  well-typed, 
once  all  the  linear  substitutions  on  one  path  are  composed  to  obtain  the  original  term. 
We  may  think  of  linear  terms  as  a  representation  which  is  only  used  internally  within 
substitution  trees,  but  not  externally  during  proof  search.  It  suffices  to  show  that 
approximate  types  are  preserved  in  substitution  trees,  and  all  intermediate  variables 
introduced  are  only  used  within  this  data-structure,  but  do  not  leak  outside.  We  will 
write  A  for  the  modal  context  A  where  all  the  type  dependencies  have  been  erased. 
Similarly,  we  write  fl  for  the  ordinary  context  T  where  all  the  dependencies  have  been 
erased. 

In  the  definition  of  higher-order  substitution  trees  we  will  distinguish  between  a 
modal  context  A  which  denotes  the  original  modal  variables  and  a  modal  context  E 
for  the  internal  modal  variables.  Similarly  to  A,  E  denotes  a  modal  context  where 
all  the  dependencies  have  been  erased.  Note  that  the  modal  variables  in  E  have  no 
dependencies  among  each  other  and  can  be  arbitrarily  re-ordered.  Moreover,  every 
internal  modal  variable  i  will  be  created  in  such  a  way  that  it  is  applied  to  all  bound 
variables,  i.e.  it  will  be  associated  with  the  identity  substitution  id.  A  higher-order 
substitution  tree  is  a  node  with  substitution  p  such  that  (A,  E)  b  p  :  (A,  S')  and  every 
child  node  has  a  substitution  pi  such  that  (A,  Ej)  b  (idA,  Pi)  :  (A,  E).  At  the  leaf,  we 
have  a  substitution  p  such  that  A  b  (idA,p)  :  (A,  E). 

Moreover,  for  every  path  from  the  top  node  po  where  (A,  Ei)  b  p0  :  (A,  E0)  to 
the  leaf  node  pn,  we  have  A  b  [pn]([pn-i]  •  •  •  Po)  :  (A,  E0).  In  other  words,  there  are 
no  internal  modal  variables  left  after  we  compose  all  the  substitutions  pn  up  to  p0. 
Moreover,  let  6  be  the  modal  substitution  corresponding  to  the  variable  definitions, 
then  the  term,  which  we  obtain  after  composing  the  modal  substitution  5  with  all  the 
substitutions  pn  up  to  po,  will  be  well-typed. 

Inserting  a  new  term  U  into  the  substitution  tree  corresponds  to  inserting  the  sub¬ 
stitution  U  —  Iq.  Before  presenting  the  algorithms  for  building  substitution  trees,  we 
discuss  our  reasons  for  adapting  substitution  tree  indexing  to  the  higher-order  setting. 
First,  we  note  that  the  order  of  term  traversal  is  not  fixed  in  advance.  For  example,  in 
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the  substitution  tree  given  earlier  we  compare  the  substitution  for  the  third  argument 
before  the  substitutions  for  the  first  argument  when  looking  up  the  term  (3)  and  term 
(4).  While  we  traverse  the  term  (1)  and  (2)  from  left  to  right.  This  feature  leads  to  very 
compact  substitution  trees  and  better  memory  usage  and  retrieval  times.  However,  in 
general  there  may  be  multiple  ways  to  insert  a  term  and  no  optimal  substitution  trees 
exist.  In  contrast  to  other  indexing  techniques  such  as  discrimination  tries,  substitution 
trees  allows  the  sharing  of  common  sub-expressions  rather  than  common  prefixes. 

This  is  especially  important  for  indexing  dependently  typed  terms.  To  illustrate 
this  point,  we  define  a  data-structure  for  lists  consisting  of  characters  and  we  keep 
track  of  the  size  of  the  list  by  using  dependent  types. 

char  :  type, 
a  :  char, 
b  :  char. 

test  :  nnn::int  .list  n  — >  type 

The  size  of  lists  is  an  explicit  argument  to  the  predicate  test.  Hence  test  takes  in 
two  arguments,  the  first  one  is  the  size  of  the  list  and  the  second  one  is  the  actual  list. 
The  list  constructor  cons  takes  in  three  arguments.  The  first  one  denotes  the  size  of 
the  list,  the  second  argument  denotes  the  head  and  the  third  one  denotes  the  tail.  To 
illustrate,  we  give  a  few  examples.  We  use  gray  color  for  the  explicit  arguments. 

test  (cons  a  (cons  3  a  (cons  2  a  (cons  1  b  nil)))) 

test  5  (cons  5  a  (cons  a  (cons  3  a  (cons  2  a  (cons  b  nil))))) 

test  6  (cons  6  a  (cons  5  a  (cons  a  (cons  3  b  (cons  2  a  (cons  1  b  nil)))))) 

If  we  use  non-adaptive  indexing  techniques  such  as  discrimination  tries,  we  process 
the  term  from  left  to  right  and  we  will  be  able  to  share  common  prefixes.  In  the  given 
example,  such  a  technique  discriminates  on  the  first  argument,  which  denotes  the  size 
of  the  list  and  leads  to  no  sharing  between  the  second  argument.  The  substitution  tree 
on  the  other  hand  allows  us  to  share  the  structure  of  the  second  argument.  The  most 
specific  linear  generalization  in  this  example  is 


list  :  char  — >  type, 
nil  :  list  0. 

cons  :  n°rz: : int  .char  — *•  list  n  — >  list  (n  +  1). 


test  *i[id]  (cons  z2 [id]  a  (cons  z3 [id]  a  (cons  *4[id]  a  (cons  z5[id]  z6[id]  nil)))). 
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This  allows  us  to  skip  over  the  implicit  first  argument  denoting  the  size  and  indexing 
on  the  second  argument,  the  actual  list.  It  has  been  sometimes  argued  that  it  is 
possible  to  retain  the  flexibility  in  non-adaptive  indexing  techniques  by  reordering 
the  arguments  to  test.  However,  this  only  works  easily  in  an  untyped  setting  and  it 
is  not  clear  how  to  maintain  typing  invariants  in  a  dependently  typed  setting  if  we 
allow  arbitrary  reordering  of  arguments.  Hence  higher-order  substitution  trees  offer  a 
adaptive  compact  indexing  data-structure  while  maintaining  typing  invariants. 

5.2  Insertion 

Insertion  of  a  term  U  into  the  index  is  viewed  as  insertion  of  the  substitution  U/iq. 
Assuming  that  U  has  type  a  in  a  modal  context  A  and  a  bound  variable  context  H, 
U/i0  is  a  modal  substitution  such  that  A  b  U/i0  :  i0::(f2i-a).  We  will  call  the  modal 
substitution  which  is  going  to  be  inserted  into  the  index  p.  It  is  often  convenient  to 
consider  the  extended  modal  substitution  A  b  \dj\,U/io  :  (A,  i0: : (Oi-or) ) .  This  will  sim¬ 
plify  the  following  theoretical  development.  Again  we  note  that  in  an  implementation, 
we  do  not  need  to  carry  around  explicitely  the  modal  substitution  idA,  but  can  always 
assume  that  any  substitution  can  be  extended  appropriately. 

The  insertion  process  works  by  following  down  a  path  in  the  tree  that  is  compatible 
with  the  modal  substitution  p.  To  formally  define  insertion,  we  show  how  to  compute 
the  most  specific  linear  generalization  (rnslg)  of  two  modal  substitutions  and  describe 
the  most  specific  linear  generalization  of  two  terms.  We  start  by  giving  the  judgments 
for  computing  the  most  specific  linear  generalization  of  two  modal  substitutions. 

(A,  Si)  b  pi  U  p2  :  S2  =>  p/{ S,  6>1;  02)  p  is  the  rnslg  of  px  and  p2 

If  (A,  Si)  b  (idA,Pi)  :  (A,  S2)  and  (A,  Si)  b  (idA,p2)  :  (A,  S2)  then  p  is  the  rnslg  of 
pi  and  p2.  Moreover,  p  is  a  modal  substitution  such  that  [idA,  Oi  ]p  is  structurally  equal 
to  pi  and  [idA,  #2 ]p  is  structurally  equal  to  p2  and  (A,  S)  b  p  :  S2.  We  think  of  pi  as 
the  modal  substitution  which  is  already  in  the  index,  while  the  modal  substitution  p2 
is  to  be  inserted.  As  a  consequence,  only  pi  will  refer  to  the  internal  modal  variables 
in  Si,  while  p2  only  depends  on  the  modal  variables  in  A.  The  result  of  the  rnslg  are 
the  modal  substitution  6\  and  02l  where  A,  Si  b  6\  :  S  and  A,  Si  b  02  :  S.  In  other 
words,  6 1  (resp.  02)  only  replaces  internal  modal  variables  in  S.  Note  that  any  modal 
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substitution  p  or  9  with  domain  E,  can  be  extended  to  a  modal  substitution  (idA,p) 
(or  (idA,  0)  resp.)  with  domain  (A,  E). 

First,  we  give  the  rules  for  computing  the  most  specific  linear  generalization  of  two 
modal  substitutions. 


(A,E)h 

(A,  SO  b  Pl  U  p2  :  S2  =►  p/ (E,  0i,  02)  (A,  Si);  Q  b  U  U  L2  :  r  =►  L/(E',  0',  0' ) 

(A,  Ei)  b  (Pl,  Lji)  U  (p2,  L2/i)  :  (S2,  i::(fihr))  =►  (p,  L/i)/((  E,  S'),  (0i,  0j),  (02,  0[>)) 

Note,  that  we  are  allowed  to  just  combine  the  modal  substitutions  6\  (02  resp.) 
and  0[  (02  resp.)  since  we  require  that  they  refer  to  distinct  modal  variables  and  all 
the  modal  variables  occur  uniquely.  Computing  the  most  specific  linear  generalization 
of  two  modal  substitutions  relies  on  hireling  the  most  specific  linear  generalization  of 
two  objects.  Note  that  we  require  that  all  objects  are  linear  higher-order  patterns 
and  are  in  quasi-canonical  form  (see  Chapter  2).  Quasi-canonical  forms  are  /ip-normal 
forms  where  the  type  information  on  the  bound  variables  in  lambda-abstraction  may 
be  omitted.  Moreover,  we  assume  that  all  modal  variables  are  lowered  and  have  atomic 
type.  We  will  use  the  spine  notation  introduced  earlier  in  Chapter  4. 

(A,  E);  Q  b  Li  U  L2  :  r  =>•  Lj (S',  0i,  02)  L  is  the  rnslg  of  L\  and  L2 

(A,  E);  Q.  b  5*1  LJ  S2  :  r  >  a  =>■  S/ (S',  9\,  92)  S  is  the  rnslg  of  Si  and  S2 

If  the  terms  L\  and  L2  have  type  r  in  modal  context  (A,  E)  and  bound  variable 

context  0,  then  L  is  the  most  specific  linear  generalization  of  Li  and  L2  such  that 
[idA,  0i \L  is  structurally  equal  to  Lx  and  [idA,  02 \L  is  structurally  equal  to  L2.  Moreover, 
0i  and  02  are  modal  substitutions  which  map  modal  variables  from  S'  to  the  modal 
context  (A,  E).  Finally,  (A,  S');  hi  b  L  :  r.  For  spines  a  similiar  invariant  holds.  If  Si 
and  S2  are  spines  from  heads  of  type  r  to  terms  of  type  a,  then  S  is  the  rnslg  of  Si  and 
S2  such  that  [idA,  0i]S  is  structurally  equal  to  Si  and  [idA,02]S  is  structurally  equal 
to  S2.  0i  and  02  are  modal  substitutions  which  map  modal  variables  from  S'  to  the 
modal  context  (A,  E). 

We  think  of  L\  (or  Si)  as  an  object  which  is  already  in  the  index  and  L2  (or  S2)  is 
the  object  to  be  inserted.  As  a  consequence,  only  Lx  (and  Si)  may  refer  to  the  internal 
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variables  in  E,  while  L2  (and  S2)  only  depends  on  A.  The  inference  rules  for  computing 
the  mslg  are  given  next. 

Normal  linear  objects 

(A,  S);  Q,  x:n  h  Lx  U  L2  :  r2  L/(S',  0U  02) 

(A,  E);0  h  Xx.Li  U  A x.L2  :  T\  — >  r2  ==>  A x.L/ (S',  6b,  02) 

■u::(<f>i-a;)  G  A 

(A,  E);0  h  u[ir]  U  u[ 7r]  :  a  =>•  w[7r]/(-,  •,  •) 

ha  G  A  i  must  be  new  L  7^  m[7t] 

(A,  E);  hi  b  u[7r]  U  L  :  a  = =>•  i[ida]/ (i::f2ha,  m[7t]/«,  L/i ) 

_ G  E _ 

(A,  E);  Q  h  i[idn]  U  L  :  a  =>  i[id^]/(i::Oi-«,  i[idn]/i,  L/i ) 

u::(<f>i-a)  G  A  L  ^  i [id]  L  ^  u[n \ 

(A,  E);h  hiU  u[tt\  :  a  = =>  i[ida]/ (i::Q  ha,  L/i ,  u[n\/i) 

(A,Y,y,n\-S1US2:r>a=i>S/(i:',di,e2) 

(A,  E);  n  h  H  ■  Si  U  H  ■  S2  :  a  =►  H  ■  S/( S',  0i,  02)  1 

Hi  ^  H2  i  must  be  new  (oUX 

(A,  E);  n  h  Hi  ■  Si  U  H2  •  S2  :  a  i[id^]/((i::fW),  Wi  ■  Si/i),  (H2  ■  S2/i ))  (  j 

Normal  linear  spines 

(A,  E);  VL  b  nil  LJ  nil  :  a  >  01  = =>•  nil/(*.  •,  •) 

(A,S);fi  h  LiUL2  :  n  =►  L/ (E1;  01?0') 

(A,  E);n  h  Si  U  S2  :  r2  >  a  =►  S/(S2,  02,0'j 

(A,  E);  hi  h  (Li,  Si)  LJ  (L2;  S2)  :  n  -  r2  >  a  =►  (L;  S')/((E1,  S2),  (01}  02),  (0' ,  0' ))  1  j 

Note  in  the  rule  for  lambda,  we  do  not  need  to  worry  about  capture,  since  modal 
variables  and  bound  variables  are  defined  in  different  context.  Rule  (*)  treats  the  case 
where  both  terms  are  existential  variables.  Note  that  we  require  that  both  existential 
variables  must  be  the  same  and  their  associated  substitutions  must  also  be  equal.  In 
rule  (1)  and  (2),  we  just  create  the  substitution  u[n\/i.  In  general,  we  would  need 
to  create  [id^]^1  (u[vr]),  but  since  we  know  that  n  is  a  permutation  substitution,  we 
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know  that  [id^p1  (7r)  exists.  In  addition,  the  inverse  substitution  of  the  identity  is 
the  identity.  Note  that  we  distinguish  between  the  internal  modal  variables  i  and  the 
“global”  modal  variables  u  in  the  rules  (la)  and  (lb).  The  key  distinction  is  that  we 
pick  a  new  internal  modal  variable  i  in  rule  (la)  while  we  re-use  the  internal  modal 
variable  i  in  rule  (lb).  This  is  important  for  maintaining  the  invariant  that  any  child 
of  (A,  E2)  b  (idA,p)  :  (A,  Ex)  has  the  form  (A,  S3)  b  (idA,p')  :  (A,  S2)  during  insertion 
(see  the  insertion  algorithm  later  on). 

In  rule  (3a)  and  (3b)  we  distinguish  on  the  head  symbol  H  and  compute  the  most 
specific  linear  generalization  of  two  objects  H\  ■  S\  and  H\  ■  S\.  If  H\  and  H2  are 
not  equal,  then  we  generate  a  new  internal  modal  variable  i[id^]  together  with  the 
substitutions  Hi  ■  S\/i  and  H2  ■  S2/i.  Otherwise,  we  traverse  the  spines  Si  and  S2  and 
compute  the  most  specific  linear  generalization  of  them.  In  rule  (4),  we  can  just  combine 
the  substitution  9i  and  92l  as  we  require  that  all  modal  variables  occur  uniquely,  and 
hence  there  are  no  dependencies  among  Ei  and  S2. 

Definition  71  (Compatibility  of  normal  objects)  Two  normal  objects  Li  and  L2 
are  incompatible,  if  (A,  E) ;  0  b  Lx  U  L2  :  r  ==>•  i[ido]/(i::fii-T,  Li/i,  L2/i).  Otherwise, 
we  call  Li  and  L2  compatible. 

In  other  words,  we  call  two  terms  compatible,  if  they  share  at  least  the  head  symbol 
or  a  A-prefix.  Moreover,  if  two  terms  are  incompatible  then  they  must  be  syntactially 
different.  Similarly,  we  can  define  the  compatibility  of  two  substitutions. 

Definition  72  (Compatibility  of  modal  substitutions) 

Two  modal  substitutions  (A,  Ex)  b  pi  :  E2  and  (A,  Ex)  b  p2  :  E2  are  incompatible,  if 
(A,  Ex)  b  pi  U  p2  :  E2  ==>•  ids/(S,p!,p2).  Otherwise,  we  call  pi  and  p2  compatible. 

As  a  consequence,  if  pi  and  p2  are  incompatible,  then  for  any  Li/i  e  pi  and 
L2/i  e  p2,  we  know  that  Li  and  L2  are  incompatible.  Next,  we  prove  that  we  can 
always  extend  modal  substitutions. 

Lemma  73 

1.  Let  E  b  a  :  S'.  If  A  b  6  :  E,  and  A  b  (9U  9)  :  (Ex,  E)  and  A  b  ( 9 , 02)  :  (E,  S2) 
then  [0]c  =  [0x,0] a  =  [0, 02]cr. 
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2.  If  A  b  0  :  E,  and  A  b  (6U  0)  :  (E1}  E)  and  A  b  ( 6 ,  02)  :  (E,  S2) 
then  [0]L  =  [01,0]L=[0,02]L. 

3.  If  A  b  0  :  E  and  Ah  (0i,0)  :  (E1;E)  and  A  b  (0,  02)  :  (E,  S2) 
then  [0]5=[01,0]5  =  [0,02]5. 

Proof:  Induction  on  the  structure  of  er,  L  and  S'.  □ 


Theorem  74  (Soundness  of  mslg  for  objects) 

1.  If  (A,  E);  ft  b  Lx  U  L2  :  t  =►  L/(E',  01}  02)  and 

(A,  E);  h  Li  :  r  and  (A,  E);  b  L2  :  r 

tden  (A,  E)  b  91  :  S'  and  (A,  E)  b  02  :  S'  and 

Li  =  [id a,  0i }L  and  L2  =  [idA,  02]L  and  (A,  S');  b  L  :  r  . 

2.  If  (A,  E);  ft  b  S1  LJ  S2  :  r  >  a  =►  S'/ (S',  01?  02)  and 

(A,  E);  ft  b  Sj  :  r  >  a  and  (A,  E);  ft  b  S2  :  t  >  a 

then  (A,  E)  b  0i  :  S'  and  (A,  E)  b  02  :  S'  and 

Sj  =  [idA,  Ail-S  and  S2  =  [idA,  02]S  and  and  (A,  S');  ft  b  S  :  r  >  a  . 


Proof:  Simultanous  induction  on  the  structure  of  the  first  derivation.  We  give  here  a 
few  cases. 


Case  V  =  (A,  E);  ft  b  A x.Li  U  A x.L2  :  rx  — >  r2  =>•  Ax.L/(E',  0i,  02) 


(A,  E);  ft,  x-.ri  b  h  U  L2  :  r2  =►  L/{ S',  01} 02) 

(A,  E) ;  O  b  Xx.Li  :  T\  — >  r2 

(A,  E);  ft,  x:T!  b  :  t2 

(A,  E) ;  O  b  \x.L2  :  Ti  — >  r2 

(A,  E);  Q,x:ti  \~  L2  :  t2 

(A,  E)  b  0!  :  S' 

(A,  E)  b  02  :  S' 

Li  =  [id A,  9i}L 
Xx.Li  =  Ax.  [idA,  Oi]L 
Xx.Li  =  [idA,  0i](Ax.L) 

A2  =  [id A,  02]L 


by  premise 
by  assumption 
by  inversion 
by  assumption 
by  inversion 
by  i.h. 
by  i.h. 
by  i.h. 
by  rule 

by  modal  substitution  definition 

by  i.h. 
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A  x.L-2  =  Aa;.[idA,  \L 
Xx.L-2  =  [idA,d2]  (Ax.L) 

(A;  S');  Q,  x\T\  b  L  :  t2 
(A;  S');  Ll  b  A x.L  :  T\  — »  r2 


by  rule 

by  modal  substitution  definition 

by  i.h. 
by  rule 


Case  V  =  (A,  S);  b  u[ 7r]  U  m[7t]  :  a  =>•  u[7r]/(-,  •,  •) 

u::($hq;)  G  A 
(A,  S);  b  u[7r]  :  a 

u[tc\  =  u[  7r] 

(A;  S)  b  •  :  • 

A;fi  b  w[7r]  :  a 


by  premise 
by  assumption 
by  reflexivity 
by  rule 
by  strengthening 


Case  D  =  (A,  S);  hi  b  u[ ix]  U  L  :  a  = =>•  z [id 

tt::(<l>i-a)  G  A 

(A,  S);0  b  u[tt]  :  a 

(A,S);Ob  L  :  a 

u[  7r]  =  [idA,u[7r]/i]i[idn] 

u[  7r]  =  m[7t] 

L  =  [idA,  V*l*[idn] 

L  =  L 

(A,  S)  b  L/i  :  i-Mha 
(A,  S)  b  u[n\/i  :  i::Q ha 
(A,  i::Q i-o;);  O  b  idj^  :  0 
(A,  i::Ll\-a);  Ll  b  i[ido]  :  a 


]/ {i\\VL\-oi,  u[n\/i,  L/i) 

by  premise 
by  assumption 
by  assumption 

by  reflexivity 

by  reflexivity 
by  rule  using  assumption 
by  rule  using  assumption 
by  definition 
by  rule 


Case  D  =  (A,  S);  ft  b  H  ■  Sl  U  H  ■  S2  :  a  =*►  H  ■  5/ (S',  6U  92) 

(A,S);flb5iUS2:r>a^  S/{  S',  6U  d2) 

(A,  S);  fib  if -Si  :  a 
(A,  S);fl  bSi  :  r  >  a 
(A,  S);  b  •  5*2  :  a 
(A,  S);flbS2:r>o: 

5'1  =  [idA,01]5' 


by  premise 
by  assumption 
by  inversion 
by  assumption 
by  inversion 
by  i.h. 
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s2  =  ^dA,e2]s 

(A,S)h01:S1 
(A,S)h02:S1 
H  ■  S\  —  H  ■  [id^lS 
H  ■  Si  —  [idA,  9i}(H  ■  S) 
H-S2  =  H •  [idA,02]S 
H-S2  =  lidA,e2](H-  S ) 
(A,S');Oh  S  :  r  >  a 
(A,£  ');n\-H-S:a 


by  i.h. 
by  i.h. 
by  i.h. 
by  rule 

by  modal  substitution  definition 

by  rule 

by  modal  substitution  definition 

by  i.h. 
by  rule 


Case  V  =  (A,  £);  Q  b  Hx  ■  Sx  U  H2  ■  S2  :  a 

(A,S);Oh  H\-  S\\  a 
(A,zy,nh  H2- S2  :  a 

H\  ■  S\  —  H\  ■  Si 
H2-S2  =  {H2-S2/i](i[idn ]) 

h2-s2  =  h2-  s2 

(A,  i: : (Oi-ck) ) ;  O  I—  idj^  :  0 

(A,  i::(ni-Q;));  Q  b  i[ido]  :a 


i[ida]/(i::Q  ha,  Hx  ■  Si/i,  H2  ■  S2/i ) 

by  assumption 
by  assumption 
by  modal  substitution  definition 
by  reflexivity 
by  modal  substitution  definition 
by  reflexivity 
by  definition 
by  rule 


Case  V  =  (A,  £);  b  (Ll5  Si)  U  (L2;  S2)  :  (n  r2)  >  a 

^(L;  £)/((£,,  S2),  (M),  (02,^)) 


(A,  S);  b  U  L2  :  n  =►  L/(E i,  0i,  02) 

(A,  E);Ob51U52:r2>a^  S/(£2,  0',  0'2 ) 

(A,  £);  Q  b  (Li;  Si)  :  Ti  t2  >  a 

(A,E);Ob  Li  :  n 

(A,  £);  b  :  r2  >  a 

(A,  £);  Q  b  (L2;  S2)  :  n  -►  r2  >  a 

(A,E);fibL2  :  n 

(A,  E);fib52:r2>a 

-^i  —  [ i d A ,  9i}L 


by  premise 

by  assumption 
by  inversion 

by  assumption 
by  inversion 

by  i.h. 
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L2  =  l*dA,  62}L 

51  =  [idA,0i]S 

52  =  [idA,d']S 
L\  —  [idA,  61, 9[}L 
L2  =  [\dA,92,9'2}L 

51  =  IidAA,W 

52  =  [\dA,92,9’2]S 

{L1;S1)  =  {[i6Al91,ef1]L;  [idA,  6U  0[]S) 
(L1]s1)  =  {idAie1A](L]S) 

(L2;S2)  —  (|idA,  02, 0yL;  [id A,02,e£S) 
(L2]S2)  =  [idA,e2,e,2](L;S) 

(A,  E)  h  (0i,9[)  :  (E!,E2)  91  and  9[  refer 

(A,  E)  h  ( d2 ,  92)  :  (Ex,  E2)  02  and  02  refer 

(A,  Si);  0  h  L  :  n 
(A,  E2);  O  h  5  :  r2  >  a 
(A,  Ex,  S2);  h  L  :  T\ 

(A,  Si,  S2);  fit-  S:r2>a 

(A,E1,S2);fih  (L;  S')  :  a 


by  i.h. 
by  i.h. 
by  i.h. 
by  lemma  73 
by  lemma  73 
by  lemma  73 
by  lemma  73 
by  rule 

by  modal  substitution  definition 

by  rule 

by  modal  substitution  definition 
to  distinct  modal  variables  (lemma  73) 
to  distinct  modal  variables  (lemma  73) 

by  i.h. 
by  i.h. 
by  weakening 
by  weakening 
by  rule 


□ 


Theorem  75  (Completeness  of  mslg  of  terms) 

1.  If  A,  E  b  9i  :  E'  and  A,  E  b  02  :  S'  and  9\  and  02  are  incompatible  and 

Li  =  [idA,  0i]L  and  L2  =  [idA,d2]L  then  there  exists  a  modal  substitution  9 *,  02, 
and  a  modal  context  E*;  such  that  (A,  E);  b  L\  U  L2  :  A  =>•  L/ (E*,  0*,  92)  and 
9\  C  9i,  02  C  02  and  E*  C  S' 

2.  If  A,  E  b  9i  :  S'  and  A,  E  b  02  :  S'  and  9\  and  02  are  incompatible  and 

Si  =  |idA,di]S  and  S2  =  [idA,#2]S  then  there  exists  a  m.odal  substitution  0*,  02, 
and  a  modal  context  E*?  such  that  (A,  E);  Q  b  Si  LJ  S2  :  A  = =>■  S/( E*,  #*,  02)  and 
9*i  C  9i,  02  C  02  and  E*  C  S'. 

Proof:  Simultanous  induction  on  the  structure  of  L  and  S. 
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Case  L  —  u[n]  and  u::& \-a  G  A 

L i  =  [id a,  0i](u[7r])  by  assumption 

Li  =  u[ 7r]  by  modal  substitution  definition 

u[ 7r]  =  u[7r]  by  inversion 

and  Li  =  u[ 7r] 

(A,  E);  b  w[7r]  U  u[7t]  :  a  ==>•  u[7r]/ (•,  •,  •)  by  rule 

•  C  E',  •  C  01?  ■  C  02 


Case  L  =  Xx.L' . 

Li  =  [idA,  01](Ax.Z/) 

Lx  =  Az.[idA  ,0i]Z/ 

L'x  =  [idA,01]L/ 
and  Lx  = 

L2  =  [idA,02](Az.Z/) 

L2  =  Ax.[idA,02]L' 

L2  =  [id A,  9q\L' 
and  L-2  =  A x.L'2 

(A,  E);  LI,  x:tx  b  L;  U  L'2  :  r2  ==►  Z//(E*,  0*,  0*) 


by  assumption 
by  modal  substitution  definition 
by  inversion 

by  assumption 
by  modal  substitution  definition 
by  inversion 

by  i.h. 


E*  C  S',  0*  C  Ox,  0 *  C  02 

(A,  E);  b  Xx.L\  U  A x.L.2  :  Tx  — >  r2  = =>-  Xx.L' / (E*,  9\,  02)  by  rule 


Case  L  =  i[idn] 

Lx  =  [idA,0i](i[idn]) 

L2  =  [idA,02](i[idn]) 

L'/i  G  0i  and  L"/i  G  d2 
L'  and  L"  are  incompatible 

Lx  =  V 
L2  =  L" 

Case:  Lx  =  u[n\  and  L2  =  L" 

(A,  E);  Ll  b  u[r\  U  L”  :  a  = =>■  i[idn]/(i::Q  b  a,  u[n\/i,  L" ji 
i::(Q  b  a)  C  S',  {u[n\/i)  C  01;  (L"/i)  C  02 
Case:  Li  =  1/  and  L2  =  u[7r] 

(A,  E);ObL'U  u[n]  :  a  ==>■  i[id^]/ b  cc,  L'/i,  u[7r]/i) 


by  assumption 
by  assumption 
by  assumption 
by  assumption 
by  modal  substitution  definition 
by  modal  substitution  definition 


by  rule 


by  rule 
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(i::(Q  b  a))  C  S',  (u[tt\/i)  C  02,  (L'/i)  C  9\ 

Case:  L1  =  H-S1  and  L2  =  H2-  S2 

Hi  ■  Si  is  incompatible  with  H2  ■  S2  and  Hi  ^  H2  by  assumption 

(A,  S);  Si  b  Hi  ■  Si  LI  H2  ■  S2  :  a  ==>■  i [id^] / (z: :0  h  a,  Hi  ■  Si/i,  H2  ■  S2/i )  by  rule 
b  a))  C  S',  (Hi  ■  Si/i)  C  6U  (H2  ■  S2/i )  C  92 

□ 

Theorem  76  (Soundness  for  mslg  of  substitutions) 

If  (A,  Si)  b  pi  U  p2  :  S2  p/( S,  9i,  6*2)  and 

(A,  Si)  b  pi  :  S2  and  (A,  Sx)  b  p2  :  S2  then  [9ijp  =  pi  and  [92\p  =  p2 
Proof:  Induction  on  the  first  derivation. 

Case  V  = _ 

(A,Si)b  - :-=►■/(-,.,•) 

■  =  ■  by  syntactic  equality 

•  =  [•](•)  modal  substitution  definition 


Case  V  =  (A,  Si)  b  (pi,  Li/i)  U  (p2,  L2/i)  :  (S2,  i::($  b  a)) 
=>(p,L/O/((S,S'),(0i,0i),(02,^2)) 


(A,  Si)  b  Pi  U  p2  :  S2  p/ (S,  9i,92) 

by  premise 

(A,  Si);  $  b  Li  U  L2  :  a  =*►  L/(S',  9[,  9'2) 

by  premise 

(A,  Si)  b  (Pi,Li/i)  :  (S2,i::(<f>  b  a)) 

by  assumption 

(A,  Si)  b  pi  :  S2 

by  inversion 

(A,  Si);  <f>  b  Li  :  a 

(A,  Si)  b  (p2,  L2/%)  :  (S2,  b  a)) 

by  assumption 

(A,  Si)  b  p2  :  S2 

by  inversion 

(A,  Si);  $  b  L2  :  a 

Li  =  [id A,  9[]L 

by  lemma  74 

L2  =  [idA,0']L 

by  lemma  74 

L\  =  [id A,  0i,  9(}L 

by  lemma  73 

L2  =  [idA,02,0']L 

by  lemma  73 

pi  =  [idA,  9ijp 

by  i.h. 

P2  =  [idA,  92\p 

by  i.h. 
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pi  =  [idA,0i,0i]p 
P2  =  [idA,  02,  9'2}p 

(. Pi,Li/i )  =  ( [id A,  0X,  0/1]p,  [idA,0i,0i]V*) 

(p2,L2/i)  =  ([idA,02,0yp,  [idA, 02,  O^jL/i) 
(pi,  Li/i)  =  [idA,  0i,  0'i](p,  L/i) 

(p2,L2/i)  =  [idA,  02, 02](p,  L/i) 

(A,S')h  p:S2 

(A,S");Oh  L  :  a 

(A,S',S")hp:S2 

(A,S',E");Oh  L  :  a 

(A,  S',  £")  h  (p,  L/i)  :  (S2,i::fi ha) 


by  lemma  73 
by  lemma  73 
by  rule 
by  rule 

by  modal  substitution  definition 
by  modal  substitution  definition 

by  i.h. 
by  i.h. 
by  weakening 
by  weakening 
by  rule 

□ 


Theorem  77  (Completeness  for  mslg  of  modal  substitutions) 

If  (A,  E)  b  6i  :  S'  and  (A,  E)  b  d2  :  S'  and  6\  and  d2  are  incompatible  and 

pi  =  [idA,0i]p  and  p2  =  [idA,d2]p  then  (A,  E)  h  pi  LJ  p2  :  Ex  p/(E*,  02)  such 

that  E*  C  S',  0J  C  0i,  0*  C  02. 


Proof:  Induction  on  the  structure  of  p. 


Case  p  =  • 

pi  =  [idA,  0i]  (-) 

pi  =  ■  and  Ei  =  • 

P2  =  [idA,  02](-) 
p2  =  ■  and  Ei  =  • 

(A,E)h-U  - 
•  C  El,  •  c  01,  .  C  e2 

Case  p  =  (p',  L/i ) 

p'l  =  NaAK  p',L/i) 

Pi  =  ([idA,  0i](p'),  [idA,  0i }L/i) 

p’i  =  (pi,Li/i) 

Pi  =  [idA,  0i]p' 


by  assumption 
by  inversion 
by  assumption 
by  inversion 
by  rule 


by  modal  substitution  definition 
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L\  —  [idA,  Qf\L 
P2  =  [idA  ,e2\{pt,L/i) 

p2  =  ([idA,  02] {p'),  [id A,  02}L/i)  by  modal  substitution  definition 

P2  =  (p2,L2/i) 

P2  =  [idA,  02\p' 

L2  =  {\AkML 

(A,  E);  $  b  Li  U  L2  :  a  = =>•  L/( E*,  01,  02)  by  completeness  lemma  75 

E*  C  S',  0*  C  0U  0 *  C  02 

(A,  E)  b  Pl  U  p2  :  E,  =►  p'/(E**,  0? ,  0 **)  by  i.h. 

E**  C  S',  Of *  C  0X,  0*2*  C  02 (A,  E)  h  (Pr,Lr/i)  U  (p2,L2/i)  :  (E1;  fii^ha) 

=►  (p',L/i)/(( S**,E*),  (0r,0J),  by  rule 

(E**,  E*)  C  S',  (0”,  0*)  C  0U  (0**,  0*)  C  02 

□ 

When  inserting  a  substitution  p2  into  a  substitution  tree,  we  need  to  traverse  the 
index  tree  and  compute  at  each  node  N  with  substitution  p  the  mslg  between  p  and 
p2.  Before  we  describe  the  traversal  more  formally,  we  give  a  more  formal  definition  of 
substitution  trees. 


NodeN  ::=  (Ehp  -»  C) 

Children  C  ::=  [N,C]  |  nil 

A  tree  is  a  node  N  with  a  modal  substitution  p  and  a  list  of  children  C.  In  general, 
we  will  write  A  b  N  :  S'  where  N  =  (Ehp  -»  C )  which  means  that  (A,  E)  h  p  :  S'  and 
all  the  children  Nt  in  C,  A  h  jVj  :  E. 

To  insert  a  new  substitution  p2  in  to  the  substitution  tree  N  where  A  h  A  :  E,  we 
proceed  in  two  steps.  First,  we  inspect  all  the  children  Nt  of  a  parent  node  N,  where 
Nt  =  Ejhpj  -»  Ci  and  check  if  pi  is  compatible  with  p2.  This  compatibility  check  has 
three  possible  results: 

1.  (A,  Sj)  h  pi  U  p2  :  S  =>  ids/ (S,  Pi,  P2)  ■ 

This  means  pi  and  p2  are  not  compatible 

2.  (A,  Ej)  h  pi  U  p2  :  E  =>■  pi/(Ej,  idSi,  02) 

This  means  p2  is  an  instance  of  p,  and  we  will  continue  to  insert  02  into  the 
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children  Ct.  In  this  case  |idA,  92}p2  is  structurally  equivalent  to  pi  and  we  will 
call  p2  fully  compatible  with  pi. 

3.  (A,  E*)  hp):Up2:S^>  p'/(E",  9ll  02) 

Pi  and  P2  are  compatible,  but  we  need  to  replace  node  Nt  with  a  new  node 
EVp'  -»  C'  where  C  contains  two  children,  the  child  node  Eji -9\  -»  Ci  and  the 
child  node  Eji- 92  -»  nil.  In  this  case  we  will  call  p2  partially  compatible  with  pt. 

This  idea  can  be  formalized  by  using  the  following  judgment  to  filter  out  all  children 
from  C  which  are  compatible  with  p2.  Moreover,  we  will  distinguish  between  the  fully 
compatible  children,  which  we  collect  in  V,  and  the  partially  compatible  children,  which 
we  collect  in  S. 

A  h  C  U  p2  :  E  =►  (V,  S) 

Fully  compatible  children  V  ::=  •  |  V,  (. N ,  9) 

Partially  compatible  children  S  ::=  •  |  S,  (N,  Ehp,  9i,  92) 

6  is  a  modal  substitution  such  that  A  h  6  :  E,  and  for  all  the  children  Cj  =  (Eji-pj  -» 
C)  in  C,  we  have  A,  E*  h  p{  :  E.  Then  V  and  S  describe  all  the  children  from  C  which 
are  compatible  with  6.  We  distinguish  three  cases. 

A  h  nil  U  <5  :  E  =>•  (nil,  nil) 

A  h  C  U  <5  :  E  =>  (V,  S)  A,E1hp1U(5:E^ids/(E,p1,(5) 

A  h  [{Yip-pi  -»  C\),C]  U  <5  :  E  =>•  (V,  S)  NC 

AhCU(5:E  ==>•  (V,  -S')  A,E1hp1U5:E^p1/(E1,idSl,02)  Pl  ±  idE 
Ah[(E1hPl^C'1),C']U(5:E=»((y,  ((E1hPl^C'1),02)),  S) 

AhPU(5:E  =>•  (V,  -S')  A,  Ex  h  pi  U  5  :  E  ==>•  p/  (S2,  9i,  92)  p  ±  Pl  ±  ids 
Ahpihpi^C'O^lU^E^^,^,  ((E1hPl^C'1),E2hP,01,02))  FC 

The  NC  rule  describes  the  case  where  the  child  Ct  is  not  compatible  with  5.  Rule 
FC  describes  the  case  where  6  is  fully  compatible  with  the  child  C\  and  the  rule 
PC  describes  the  case  where  5  is  partially  compatible  with  Ci.  Before  we  describe 
the  traversal  of  the  substitution  tree,  we  prove  some  straightforward  properties  about 
these  rules 
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Lemma  78 

If  A  b  C  U  d  :  E  =>•  (V,  S)  and  A  b  <5  :  E  and  for  any  (Ejhpj  -»  C')  G  C  with 
A,  E i  b  pi  :  E  f/ien 

1.  for  any  (Ni:  02 )  G  V  where  Ni  =  (Ej  b  p*  -»  Cj)?  we  /iaue  [02]  p*  =  <E 

2.  for  any  (Nu  S'  b  p' ,9\,92)  &  S  where  Ni  =  (E,:  b  p,:  -»  Cj);  we  have  \0-f\p'  =  5 

and  [0i ]p'  =  pi. 

Proof:  By  structural  induction  on  the  first  derivation  and  by  using  the  previous  sound¬ 
ness  lemma  for  rnslg  of  substitutions  (lemma  76). 

Case  V  = _ . 

A  b  nil  U  5  :  E  =>•  (nil,  nil) 

Trivially  true. 

Case  p  =  AbCUi:E=>(V,5)  A,  E,  b  Pl  U  6  :  E  =►  ids/(E,  Pl,  8)  ^ 

A  b  [(Eih-pi  -»  CO,  C\  U  5  :  E  =►  (V,  S) 

By  i.h.,  for  any  (N,92)  G  V,  Ni  =  (Ej  b  pi  -»  Cf),  we  have  \02\pi  =  5  and  for 
any  (Ni,  E'  b  p' ,  9[,  0'2)  G  S  where  Ni  =  (Ej  b  pj  -»  Cf),  we  have  \9'2 \p'  =  d  and 
WiW  =  Pi- 

Case  •D  =  AhC'U5:S^(^>S)  A,  E,  b  px  U  5  :  E  =►  p1/(E1,  idSl,  02) 

A  b  [(Eii-p!  -»  CO, ~C\  U  <5  :  E  ((V  ,  (Eii-p!  -»  CO),  5)  F° 

By  i.h.,  for  any  (iVj,02)  G  V,  Ni  =  (Ej  b  pi  -»  Cf),  we  have  \92\pi  =  5  and  for 
any  (iVj,  (S'  b  p',9[,9 '2))  G  S  where  iVj  =  (Ej  b  pi  -»  Cf),  we  have  I92jp'  =  5 
and  \9'f\p'  =  Pi.  By  soundness  lemma  76,  [02]pi  =  d,  therefore  for  any  (Ni,9')  G 
(V,  ((Sihpi  -»  Ci),02)),  where  N  =  (Ej  b  pt  -»  Cf)  we  have  [9'Jpi  =  d. 

Case  V=  AbCULE^(b,5)  A,  E,  b  px  U  5  :  E  =►  p*/(E2,  9X,  02) 

A  b  [(En-p!  -»  Cf),  C\  U  <5  :  E  =►  (V,  (S  ,  ((E ^Pl  -»  Cx),  S2h p*,  9U  02))  ° 

By  i.h.,  for  any  (Ni,92)  G  V,  Ni  =  (Ej  b  pj  -»  Cf),  we  have  [02]p,:  =  d  and  for 
any  (Ni,  (S'  b  p' ,  9[,  02))  G  S  where  Nt  =  (Ej  b  pj  -»  Cf),  we  have  [02]p'  =  5  and 
[011//  =  pi.  By  soundness  lemma  76,  [02]p*  =  d  and  [0i]p*  =  pi,  therefore  for 
any  (Nh  E'  b  P',01,0')  G  (S  ,  ((E^p,  -»  Cx),  E2hp*,  01}  02)),  where  IVj  =  (Ej  b 
Pi -»  Cf)  we  have  [01]  p'  =  pj  and  [01]p'  =  5. 
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□ 

Next,  we  show  insertion  of  a  substitution  5  into  a  substitution  tree  N.  The  main 
judgment  is  the  following: 

A  b  N  U  5  :  E  =>•  N '  Insert  5  into  the  substitution  tree  N 

If  N  is  a  substitution  tree  and  5  is  not  already  in  the  tree,  then  N'  will  be  the 
new  substitution  tree  after  inserting  6  into  N.  We  write  [N'/N^C  to  indicate  that 
the  i-th  node  Nt  in  the  children  C  is  replaced  by  the  new  node  N'.  Recall  that  the 
substitution  <5  which  is  inserted  into  the  substitution  tree  N  does  only  refer  to  modal 
variables  in  A  and  does  not  contain  any  internal  modal  variables.  Therefore,  a  new  leaf 
with  substitution  5  must  have  the  following  form:  -h 6  -»  nil.  Similarly,  if  we  split  the 
current  node  and  create  a  new  leaf  -\-92  — »  nil  (see  rule  “Split  current  node”). 

Create  new  leaf 

AhCU<5:  £'=►  (-,•) 

A  h  (E cp  -»  C)  U  5  :  S'  =*►  (E cp  -»  (C,  (-m$  -»  nil)) 


Recurse 

AhCu5:S'^(b,5)  NteC  (Ni,e2)e  v  A  h  N  U  e2  N’ 

A  b  (EVp  -»■  C)  U  <5  :  E  (SVp  -»  \N'/Ni]C 

Split  current  node 

AhCuTS^(-,5)  NteC  Ni  =  (Eji-p*  -»  Ci)  (Nu  SVp,  9U  02)  e  S 
A  h  (S Vp  C)  U  5  :  E  =*►  (EVp  [(E*hp  -»  ((E<i-0i  Q),  (-h 92  nil )))/Ni\C) 

The  above  rules  always  insert  a  substitution  <5  into  the  children  C  of  a  node  Ehp 
C.  We  start  inserting  a  substitution  L/i0  into  the  empty  substitution  tree  which 
contains  the  identity  substitution  i0[id]/*o  and  has  an  empty  list  of  children.  After  the 
first  insertion,  we  obtain  the  substitution  tree  which  contains  the  identity  substitution 
*o[id]/*0  and  the  child  of  this  node  contains  the  substitution  L/i0.  In  other  words, 
we  require  that  the  top  node  of  a  substitution  tree  contains  a  redundant  identity 
substitution  which  allows  us  to  treat  insertion  of  a  substitution  5  into  a  substiution  tree 
uniformly.  This  leads  us  to  the  following  soundness  statement  where  we  show  that  if  we 
inserte  a  substiutition  5  into  the  children  (7,  then  there  exists  a  child  Ci  =  Ejh p*  C[ 

in  C  and  a  path  from  p,:  to  pn,  where  pn  is  at  a  leaf  such  that  [pn][Pn-i]  ■  ■  ■  pi  —  S. 
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Theorem  79  (Soundness  of  insertion) 

If  Ah  (EVp'  -»  C)U5  :  E  ==>•  (EVp'  -»  C')  then  there  exists  a  child  Ci  =  (Eji -pi  -»  C') 
and  a  path  from  pi  to  pn  such  that  [pn][Pn-i] . . .  Pi  —  S. 


Proof:  By  induction  on  the  first  derivation  using  the  previous  lemma  78.  □ 


We  illustrate  this  process  by  an  example.  Assume  we  have  the  following  two  modal 
substitutions: 


Pi  =  1(9  «)Ad  /  (Ax.w[id])/i2,  (h  c)/z3] 
5  =  l (9  b)/h,  c/i 2,  (h  a)/i3] 


Then,  the  most  specihc  linear  common  generalization  of  the  substitution  is  com¬ 
puted  as  follows.  We  compute  the  most  specihc  linear  generalization  for  each  ele¬ 
ment.  Hence  the  resulting  most  specihc  linear  common  substitution  p  of  p\  and  6  is 
[(<7  i4[id])/i1,  i2[id]/i2,  (h  i5[id])/i3],  In  addition,  p\  will  be  split  into  the  most  specihc 
linear  common  substitution  p  and  the  substitution  9i  =  [a/i4,  /  (Ax.u[id])/i2,  c/i5], 
s.t.  \6i\p  =  p\.  Similarly  S  will  be  split  into  p  and  the  substitution  #2  =  [6/i4,  c/i2,  «/*s], 
s.t.  [02]p  =  d.  Next,  we  illustrate  the  process  of  inserting  the  terms  (1)  to  (4)  from 
page  147  into  a  higher-order  substitution  tree. 
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pred  (h  (g  b))  (g  b)  a 

true 

(1) 


insert 

pred  (h  (g  b))  (g  b)  b 

> 


insert 

pred  (h  (h  a))  (g  b)  (f  iax  x.  E 


pred  (h  il  id  ) 

o 

II 

"O 

s/ 
s  \ 

/ 

(9  b)  =  il 

(h  b)  =  il 

i2id  =  i2 

/  \ 

(f  lax  x.  E  x) 

/ 

a  =  i2 

\ 

b  —  i2 

El  unit 

(3) 

true 

true 

(1) 

(2) 

In  the  second  step,  we  can  see  why  it  is  important  to  re-use  the  name  of  internal 
modal  variables  when  computing  the  mslg  of  two  terms.  Here  we  need  to  compute  the 
rnslg  of  pred  (h  (g  b))  (g  b)  i2[id]  and  pred  (h  (h  b))  (g  b)  (f  Arc.wfid]).  The  result  is 
the  substitution  0\  =  |(g  b)/ii,  i2[id]/i2]  and  02  =  [(h  b )/ii,  (f  Aar.w[id])/?‘2J.  Since  we 
chose  to  re-use  the  variable  i2  and  created  a  substitution  i2[id]/i2,  we  can  add  a  new 
node  with  the  substitution  02  and  children  a  =  i2  and  b  =  i2  into  the  tree  without 
violating  the  stated  invariant  or  renaming  the  children. 

Finally,  we  note  that  we  may  need  to  apply  weakening  before  inserting  a  substitu¬ 
tion  6  into  the  substitution  tree  N.  Consider  the  first  two  insertions  into  the  empty 
substitution  tree  in  the  previous  example.  Since  the  first  two  terms  inserted,  do  not 
refer  to  any  modal  variables  A  =  •.  In  other  words,  if  the  top  node  in  the  substitution 
tree  is  well-defined  in  the  empty  modal  context  and  A  =  •.  In  the  third  step,  we  are 
inserting  the  object  pred  (h  (h  b))  (g  b)  (f  Aa;.w[id])  which  refers  to  the  modal  variable 
it.  Hence  before  inserting  the  third  term,  we  need  to  weaken  the  modal  context  A  such 
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that  such  that  A  b  N  :  £  and  A  b  5  :  £. 


5.3  Retrieval 


In  general,  we  can  retrieve  all  terms  from  the  index  which  satisfy  some  property.  This 
property  may  be  finding  all  terms  from  the  index  which  unify  with  a  given  term  U  or 
finding  all  terms  V  from  the  index,  s.t.  a  given  term  U  is  an  instance  or  variant  of 
V.  One  advantage  of  substitution  trees  is  that  all  these  retrieval  operations  can  be 
implemented  with  only  small  changes.  A  key  challenge  in  the  higher-order  setting  is 
to  design  efficient  retrieval  algorithms  which  will  perform  well  in  practice.  In  Chapter 
3  we  presented  a  unification  algorithm  for  linear  higher-order  patterns.  We  specialize 
this  algorithm  to  check  whether  the  term  L-2  is  an  instance  of  the  term  L±.  We  treat 
again  internal  modal  variables  differently  than  global  modal  variables.  In  addition, 
both  terms  must  be  linear  unlike  the  algorithm  in  Chapter  3  where  only  one  of  them 
has  to  be.  The  principal  judgement  are  as  follows: 


(A,S);Oh  L1 
(A,£);flbSi 


L2  :  t/(0,p)  L2  is  an  instance  of  Li 

S2  :  t  >  a/(6,p)  S2  is  an  instance  of  S2 


Again  we  assume  that  L\  and  L2  must  be  well-typed  in  the  modal  context  A,  £  and  the 
bound  variable  context  Q.  We  assume  that  only  Li  contains  internal  modal  variables 
and  is  stored  in  the  index,  while  L2  is  given,  and  that  the  modal  variables  occurring  in 
L\  are  distinct  from  the  modal  variables  occuring  in  L2.  This  implies  that  (Ai,  £);  h 
Li  :  t  and  A2;fi  b  L2  :  r  and  A  =  (Lambda i,A2).  p  is  the  substitution  for  some 
internal  modal  variables  in  £  while  9  is  the  substitution  for  some  modal  variables  in 
Ai.  Both  substitutions  can  always  be  extended  with  identity  substitutions  such  that 
the  extension  of  6  (e.g.  p)  covers  all  the  variables  in  A  (e.g.  £).  Moreover,  |idA2, 0, p]Ti 
is  syntactically  equal  to  L2. 
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_ E  E _ 

(A,  E);  fi  h  i[ido]  =  L  :  a  /  (•,  (L/i)) 


existsL  —  1 


_ u::$ha  E  A _ 

(A,  E);  fi  h  u[ti\  =  L  :  a  /  (([7r]_1  L/u ),  •) 


existsL  —  2 


(A,  E);  12,  x:ti  h  L1  =  L2  :  r2  /  (0,  p) 

(A,  E);  fi  h  Ax.Li  =  Xx.L2  :  Ti  — >  T2  /  (0,  p) 
(A,  E);  fill-  Si  =  S2:r>a/(0,p) 
(A,  E);  fi  h  A-  •  51  =  H  ■  S2  :  a  /  (0,  p) 


(A, 52);0  lh  nil  =  nil  :  ce  >  a;  /  (•,•) 

(A,E);fi  h  L1  =  L2  :  n  /  (gi^pQ  (A,S);fi  lh  Sx  =  S2:  t2>  oc  /  (92,  p2) 

(A,  E);  fi  lh  (Li,  Si)  =  (L2,  S2)  :  T\  — >  t2  >  a  /  ((^i,  d2),  (pi,  p2)) 

Note  that  we  need  not  worry  about  capture  in  the  rule  for  lambda  expressions 
since  existential  variables  and  bound  variables  are  defined  in  different  contexts.  In 
the  rule  app,  we  are  allowed  to  union  the  two  substitutions  0\  and  d2,  as  the  linearity 
requirement  ensures  that  the  domains  of  both  substitutions  are  disjoint.  Note  that  the 
case  for  matching  an  existential  variable  u[n\  with  another  term  L  is  simpler  and  more 
efficient  than  in  the  general  higher-order  pattern  case.  In  particular,  it  does  not  require 
a  traversal  of  L  (see  rules  existsL- 1  and  existsL-2 ).  Since  the  inverse  of  the  substitution 
7 r  can  be  computed  directly  and  will  be  total,  we  know  [t]-1  L  exists  and  can  simply 
generate  a  substitution  [t]  1  L/u.  The  algorithm  can  be  easily  specialized  to  retrieve 
variances  by  requiring  in  the  existsL-2  rule  that  L  must  be  u[n\.  To  check  unifiability 
we  need  to  add  the  dual  rule  to  existsL-2  where  we  unify  L  with  an  existential  variable 
u[n\.  The  only  complication  is  that  L  may  contain  internal  modal  variables  which  are 
defined  later  on  the  path  in  the  substitution  tree. 

The  instance  algorithm  for  terms  can  be  straightforwardly  extended  to  instances  of 
substitutions.  We  define  the  following  judgment  for  it: 

A,  E  b  pi  =  p2  :  E'/ ( 6 ,  p)  p2  is  an  instance  of  pi 

We  assume  that  pi  and  p2  are  modal  substitutions  from  a  modal  context  S'  to  a 
modal  context  A,  E.  p  is  a  modal  substitution  for  the  modal  variables  in  E,  while  9  is 
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the  modal  substitution  for  the  modal  variables  in  A  such  that  \6,  crjpi  is  syntactically 
equal  to  p2. 

(A,S)h-  =  ■:•/(•,•) 

(A,  Si)  b  p1  =  p2  :  £ 2/(9,  p)  (A,  £,);  Q  b  Lx  =  L2  :  r/(0',  p') 
(A.SOMPbM)  =  (p2,^2A):(S2,u:(fihT))/((0,d'),(p,p/) 

Now  we  show  soundness  and  completeness  of  the  retrieval  algorithm.  We  first  show 
soundness  and  completeness  of  the  instance  algorithm  for  terms. 


Theorem  80  (Soundness) 

1.  If  (A,  £);Q  b  L\  =  L2  :  r/(9,p )  for  some  Ai  and  A2  where  A  =  (Lambdai,  A2) 

and  (Ai,  £);  b  Li  :  r  and  A2;  b  L2  :  r  £/ien  [idA2,  p]Ti  =  L2. 

2.  If  (A,  £);f2  b  =  S2  :  r  >  a/(9,p )  /or  some  Ai  and  A2  where  A  = 

(Lambda^,  A2)  and  (A1;£);r2  b  5/  :  r  >  a  and  A2;  b  5'2  :  r  >  «  £/ien 

|idAa,MSi  =  S2. 


Proof:  Simultanous  structural  induction  on  the  hrst  derivation.  The  proof  is  straigh- 
forward,  and  we  give  a  few  cases  here. 


Case  V 


(A,£);f2bi[idn]  =  L  :«/(-,  (L/i)) 


existsL  —  1 


i\:Qha]  £2  b  i[idn]  :  a 
A2;  O  b  L  :  a  and  A  =  A2 
L  =  L 

[idA2,L/i](i[idn])  =  L 


by  assumption 
by  assumption 
by  reflexivity 
by  substitution  definition 


Case  V 


u::Qha  G  A  . 

_  C>rf'rf  CT  C  /  _  / 

(A,  £);  b  u[ 7r]  =  L  :  a  /  (([vr]_1  L/u),  •) 


b  u[ix\  :  a 

A2;  O  b  L  :  a  and  A  = 


1  L)  =  L 

IidA2,[7r]_1L/u](u[7r]) 


A2,  u::&\-a 


=  L 


by  assumption 
by  assumption 
by  property  of  inversion 
by  substitution  definition 
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Case  V 


(A,  E);0,x:ri  b  Lx  =  L2  :  t2  /  (0,p) 

(A,  E);  12  b  Xx.Li  =  A x.L2  :  Ti  — >  T2  /  (0,  p) 


lam 


(A1;  E);  12  b  Ax. Li  :  Ti  — >  r2 
(Ai,  E);  12,x:ti  b  Lx  :  r2 
A2;  12  b  Ax.L2  :  r\  — >  r2 
A2, ;  12,x:ti  b  L2  :  t2 
[idA2,0,p]L1  =  L2 


by  assumption 
by  inversion 
by  assumption 
by  inversion 
by  i.h. 


Cage  v  =  (A,  E);  12  b  U  =  L2  :  n  /  (A,  E);  Q  lb  S1  =  S2  :  r2  >  a  /  (02,  p2) 

(A,  E);  12  lb  (Lx;  Si)  =  (L2;  S2)  :  Ti  — >  r2  >  a  /  ((#1,  02),  (pi,  p2)) 


(A1;E);fib(L1;51):r>a 
(Ai;  E);  Ob  L\  \  T\ 

(Ai;  E);  12  b  Si  :  n  — >  r2  >  a 
A2;  12  b  L-2  :  Ti 
A2;  12  b  S2  :  Ti  — >  t2  >  a 
[idA2, 9i,  pi]Li  =  L2 

[idA2,  6*2,  pi,  p2]Li  =  L2 
[idA2,0i,02,Pi,P2]5'i  = 

[idA2, 0i,  02, Pi, P2K-S1  <Si)  =  [idA20i,  02, Pi, P2K-S2  $2) 


by  assumption 
by  inversion 

by  inversion 


by  i.h. 
by  i.h. 
by  weakening 
by  weakening 
by  rule 

and  substitution  definition 


□ 


Next,  we  show  soundness  of  retrieval  for  substitions. 

Theorem  81  (Soundness  of  retrieval  for  substitutions) 

If  (A,  E)  b  pi  =  p2  :  E'/ ( 9 ,  p)  and  (Ai,  E)  b  pi  :  S'  and  A2  b  p2  :  S'  and  (Ai,  A2)  =  A 
and  all  the  variables  in  E,  Ai  and  A2  are  distinct  then  [idA2,0,p]pi  =  p-2- 

Proof:  Structural  induction  on  the  first  derivation  and  using  previous  lemma  80.  □ 

For  completeness  we  show  that  if  the  term  L2  is  an  instance  of  a  linear  term  L  then 
the  given  algorithm  will  succeed  and  return  substitution  6*  for  the  modal  variables 
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and  a  substitution  p*  for  the  internal  modal  variables  ocurring  in  L.  This  establishes  a 
form  of  local  completeness  of  the  given  retrieval  algorithm.  We  will  show  later  a  global 
completeness  theorem,  which  states  that  any  time  we  compute  the  rnsgl  of  a  term  Li 
and  L2  to  be  L,  then  we  can  show  that  L2  is  in  fact  an  instance  of  L.  More  generally, 
we  show  that  any  time  we  insert  a  substiution  L2/io  we  can  also  retrieve  it. 

Theorem  82  (Completeness  of  instance  algorithm  for  terms) 

1.  If  (A1?E);f2  b  L  :  r  and  A2;  hi  b  L2  :  r  and  A  =  (Ai,A2)  and  A  b  6  :  Ai  and 
A  b  p  :  E  and  [idA2, 0,  p\L  =  Lx  then  (A,  E);  Q  b  L  =  L2  :  r/(6*,  p*)  where  9*  C  9 
and  p*  C  p. 

2.  If  (Ai,  E);  Q  b  S  :  r  >  a  and  A2;  b  S2  :  t  >  a  and  A  =  (Ai,  A2)  and  A  b  9  :  Ai 
and  A  b  p  :  E  and  [idA2,  d,  pjS1  =  S2  then  (A,  E);f2  b  S  =  S2  :  r  >  a/(9*,p*) 
where  9*  C  9  and  p*  C  p. 

Proof:  Simultanous  structural  induction  on  the  first  typing  derivation. 

Case  V  =  h  L  ■  r2 

(A1;  E);  Q  b  A x.L  :  T\  — >  r2 


A2;  b  A x.L2  :  Ti  — >  r2 

A2;  tt,x:Ti  b  L2  :  r2 

|idA2,6>,p](Ax.L)  =  A  x.L2 

Ax. [id  a2,6>,p](L)  =  Ax.L2 

[idA2,0,p](A)  =  L2 

(A,  E);  x\T\  b  L  =  L-2  \  r2/ (0*,  p*) 

9*  C  9  and  p*  C  p 

(A,  E) ;  O  b  A  x.L  =  A  x.L2  :  T\  — >  r2/ (d*,  p*) 

Case  P  =  £  E _ 

(Ai,  E);  Q  b  i[idQ]  :  a 


by  assumption 
by  inversion 
by  assumption 
by  substitution  definition 
by  syntactic  equality 
by  i.h. 

by  rule 


by  rule 


i::fl ha;  b  i[ida]  :  a 
E  =  Ei,  i::Q \-a,  E2 
A2;  b  L2  ol 
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IidAa,0,pMidn])  =  L2 

L'/i  e  p 

L'  =  L2  and  L2/i  e  p 

(A,  Q  b  i[idn]  =  L2  :  a/ (•,  L2/i) 

■  Q  id A  and  ( L2/i )  C  p 


by  assumption 
by  assumption 
by  substitution  definition 
by  rule 


Case  V 


urA &ha  G 

(Ai,  E);  Q  b  u[ 7r]  :  a 


b  u[n]  :  a 
Ai  =  A[,  u::$\-a,  A" 

A2;  b  L2  :  « 

9  =  (91,L/u,92) 

[idA2,0,p](u[7r])  =  L2 

[7 t]L  =  L2 

L  =  [7r]  1  L-2  and  [7r]  ( [7r]  ^ 1  L2 )  =  L-2 

A2,  u:\Q\-a;  b  u[n\  =  L2  :  a/Qvrp1  L2/u,  •) 

(  [t] — 1  L2  / u)  C  d  and  •  C  p 


by  rule 

by  assumption 
by  assumption 
by  assumption 
by  substitution  definition 
by  inverse  substitution  property 

by  rule 


Case  v  =  (A1,E);^bL1  :n  (A1;  S);  Q  lb  Si  :  n  ->  r  >  a 

(Ai,  E);  Q  lb  (Li,  Si)  :  t  >  a 


[id  A2,6,pl(L1;S1)=S' 
[id^MW;  [idAa,0,p](51)  =  5/ 
S'  =  (L2]S2) 

[id  A2,9,pj(L1)  =  L2 
[id  Aa,0,p](S1)  =  S2 
A2;  O  b  (L2;  S2)  \  t  >  a 
h2)  b  L2  :  T\ 


by  assumption 
by  substitution  definition 
by  inversion 
by  inversion 
by  inversion 
by  assumption 
by  inversion 


A2;  b  62  :  tl  — >  t  >  a 

(A,  E);Q  b  L\  =  L2  :  ri/(d[,p[)  and  9^  C  9  and  p\  C  p 

(A,  E);  Q  b  Si  =  S2  :  T\  — »  r  >  ct/ (9%,  p2)  and  9\  C  9  and  p[j  C  p 
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(A,  E);  Ol-Li  =  L2:r1/(9*1,p*1) 

(A,  E);  Q  b  Si  =  S2:t1^t>  a/ (92,  p2) 

(A,  E);  Q  b  (Li;  Si)  =  ( L2;S2):r>a/m,e*),(pl,p*2 )) 

(61,9*)  Cfland  {p\,p*2)Qp 

□ 


by  weakening 
by  weakening 
by  rule 
by  subset  property 


Next,  we  show  the  global  completeness  of  the  rnslg  and  instance  algorithm.  We 
show  that  if  the  rnslg  of  object  L\  and  L2  returns  the  modal  substitutions  9 1  and  d2 
together  with  the  rnslg  L ,  then  in  fact  the  retrieval  algorithm  shows  that  L\  is  an 
instance  of  L  under  9\  and  L2  is  an  instance  of  L  under  02.  This  guarantees  that  any 
time  we  insert  a  term  L2  we  can  in  fact  retrieve  it. 

Theorem  83  (Interaction  between  mslg  and  instance  algorithm) 

1.  If  for  some  Ax  and  A2  where  (A^E);!!  b  Li  :  r  and  A2;  b  L2  :  r  and  A  = 
(A1;  A2)  and  (A,  E);  Q  b  Lx  U  L2  :  r  =>•  L/(E',  9i,  92)  then 
((A,E),E');Ob  L  =  Lx  :  r/(9' ,9X)  and 

((A,  E),E');ObL  =  L2:t/(9",92)  and  9'  C  id (AjS)  and  9"  C  id(A,s)- 

2.  If  for  some  Ai  and  A2  where  (A1;  E);HbSli>a:r  and  A2;  0  b  S2  :  t  >  a  and 
A  =  (Ax,  A2)  and  (A,  E);  Q  b  5*1  U  S2  :  r  >  a  =>-  S/ (S',  9i,  92)  then 

((A,  E),E');flb5  =  S1:t>  a/(9’,  9J  and 

((A,  E),  S');  It  b  S  =  S2  :  t  >  a/(9" ,  92)  and  9' C  and  9”  C 

Proof:  Simultanous  structural  induction  on  the  first  derivation. 

Case  V  =  (A’  X:Tl  h  Li  u  L2  ■  r2  =>  L/(E',  0i,  02) 

(A,  E);  b  Xx.Li  U  A x.L2  :  T\  — >  r2  =>•  Ax.L/(E',  9i,  92) 


(A,  E);  x:ti  b  L  =  L\\  r2/ (0',  0i)  and  9'  C  id(A,s)  by  i.h. 

(A,  E);fib  A x.L  =  Xx.Li  :  T\  — >  T2j (9',  9\)  by  rule 

(A,  E);  Q,  x:ti  b  L  =  L-2  :  r2/ ( 9",  92)  and  9"  C  id(A)E)  by  i.h. 

(A,  E);fib  A  x.L  =  X  x.L2  :  T\  — »  r2/ (0",  02)  by  rule 
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Case  V  = _ e  A _ 

(A,  E);  Q  b  u[ 7r]  U  u[n]  :  a  = =>-  u[ 7r] / ( • ,  •,  •) 

[Vp1  (m[7t])/m)  =  (u[id$]/w)  =  idAl  where  Ai  =  u::$\-a 

A,  E;  fl  b  u[tt]  =  u[n }  :  a/(w[id$]/w,  •)  and  u{\d<$>/u)  C  id(AjE)  by  rule  existsL  —  2 


Case  V  = 


uw&ha  e  (A,  E)  i  must  be  new 
(A,  E);  fl  b  u[ 7r]  U  L  :  a  = =>-  i[id^]/ -ufvrj/i,  L/i) 


A,E;fih  i[idn]  =  up]  :  «/(-,  up]/i)  and  •  C  id(AiS) 
A,E;Oh  i[idn]  =  L  :  a/ (•,  L/i)  and  •  C  id(A)E) 

Case  Cases  for  rules  (lb)  and  (2)  follow  the  same  pattern. 


by  rule 
by  rule 


Case  V  —  P  /  P  i  must  be  new 

(A,  E);  12  b  Hx  ■  S1  U  H2  •  S2  :  a  i[idn]/((i::fii-a),  {Hi  •  SJi),  (H2  ■  S2/i )) 


A,  E;  VL  b  i[idp  =  H±  ■  Si  \  a/ (•,  Hx  ■  Si/i)  and  •  C  id(AjS) 
A,  E;  Q,  b  i[idp  =  H2  ■  S2  :  a/ (•,  H2  ■  S2/i)  and  •  C  id(AjS) 


by  rule 
by  rule 


Case  V  =  (A,  E);  Vi  b  (Li;  Si)  LJ  (L2;  S2)  :  n  -»•  t2  >  a 


(L;S)/((Ei,E2)P0iAM«)) 


(A,  E);  b  Li  U  L2  :  n  =►  U/( Els  0i,  0') 

(A,  E);  b  Si  U  S2  :  r2  >  a  =►  S/(S2,  62,  0' ) 

(A,  E);  fl  b  L  U  Li  :  Ti/{0\,  6\)  and  9\  C  id(AjE) 

(A,  E);  Q  b  L  U  S2  :  n/^,  02)  and  9*2  C  id(A)S) 

(A,  E);  n  b  S  LJ  Si  :  r2  >  a/{9{\  9[)  and  9\*  C  id{A)S) 
(A,  E);  n  b  S  LJ  S2  :  r2  >  a/(dp,  0'2)  and  9**  C  id{A)S) 
(A,  E);fib  (L;  S)  U  (Lp  Si)  :  n/((^,  0?*),  *i)) 

(A,  S);  fl  b  (L;  S)  U  (L2;  S2)  :  n/{{9*,  9**),  (92,  9'2)) 

(9*,  9**)  C  id(AiE)  and  (0^*)  C  id(A,s) 


by  inversion 

by  i.h. 
by  i.h. 
by  i.h. 
by  i.h. 
by  rule 
by  rule 


177 


CHAPTER  5.  HIGHER-ORDER  TERM  INDEXING 


Theorem  84  (Interaction  of  insertion  and  retrieval  for  substitutions) 

If  (A,  E)  b  Pl  Up2  :  S'  =►  p/(E",  Oi,  02)  then  (A,  E),  E"  b  p  =  Pl  :  E'/(0',0i)  and 
(A,  E),  E"  b  p  =  p2  :  E'/ (0",  02)  and  6'  C  id(A,s)  and  and  Q  id(A,s)- 

Proof:  Structural  induction  on  the  first  derivation  and  use  of  lemma  83.  □ 

Next,  we  show  how  to  traverse  the  tree,  find  a  path  [pn][p„_r]  . . .  pi  such  that  is 
an  instance  of  it  and  return  a  modal  substitution  6  such  that  [0][pn][Pn-i]  ■■■  Pi  —  P2- 
Traversal  of  the  tree  is  straightforward. 

A,  E  b  p  =  p2:  E'/(0',  p')  A  b  C  =  p'  :  E/0 
“  A  b  [(Sep  C),  C'\  =  p2:  S7(0',0) 


there  is  no  derivation  such  thatA,  E  bp  =  p2  :  E//(0',p/) 

A  b  C'  =  p  :  E/0 

A  b  [(Ehp-»  C),C]  =  p2  :  S'/0 


Theorem  85  (Soundness  of  retrieval) 

If  A  b  C  =  p'  :  E'/0  then  there  exists  a  child  Ci  with  substitution  p%  in  C  such  that  the 

path  [0][pn][Pn-i]  •  •  •  [/h]  =  ft- 


Proof:  By  structrural  induction  on  the  hrst  derivation  and  use  of  lemma  81.  □ 

Finally,  we  show  that  if  we  insert  p  into  a  substitution  tree  and  obtain  a  new  tree, 
then  we  are  able  to  retrieve  p  from  it. 

Theorem  86  (Interaction  between  insertion  and  retrieval) 

If  A  b  (E  b  p  -»  C)  U  p2  :  E  (E  b  p  -»  C')  then  A  b  C  ==  p2f  idA- 

Proof:  Structural  induction  on  the  derivation  using  lemma  84.  □ 

5.4  Experimental  results 

In  this  section,  we  discuss  examples  from  three  different  applications  which  use  the 
tabled  logic  programming  engine  in  Twelf.  Here  we  focus  on  an  evaluation  of  the 
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indexing  technique.  For  a  more  in  depth-discussion  of  tabling  and  these  examples  we 
refer  the  reader  to  the  previous  chapter  4.  All  experiments  are  done  on  a  machine  with 
the  following  specifications:  1.60GHz  Intel  Pentium  Processor,  256  MB  RAM.  We  are 
using  SML  of  New  Jersey  110.0.3  under  Linux  Red  Hat  7.1.  Times  are  measured  in 
seconds.  All  the  examples  use  variant  checking  as  a  retrieval  mechanism.  Although  we 
have  implemented  subsumption  checking,  we  did  not  observe  substantial  performance 
improvements  using  subsumption.  A  similar  observation  has  been  made  for  tabling  in 
the  first-order  logic  programming  engine  XSB  [59].  Potentially  subsumption  checking 
becomes  more  important  in  theorem  proving,  as  the  experience  in  the  first-order  setting 
shows. 


5.4.1  Parsing  of  first-order  formulae 

Parsing  and  recognition  algorithms  for  grammars  are  excellent  examples  for  tabled 
evaluation,  since  we  often  want  to  mix  right  and  left  recursion  (see  also  [68]).  In 
this  example,  we  adapted  ideas  from  Warren  [68]  to  implement  a  parser  for  first-order 
formulas  using  higher-order  abstract  syntax  (see  also  Chapter  4). 


#tok 

noindex 

index 

reduction 

in  time 

improvement 

factor 

20 

0.13 

0.07 

46% 

1.85 

58 

2.61 

1.25 

52% 

2.08 

117 

10.44 

5.12 

51% 

2.03 

178 

32.20 

13.56 

58% 

2.37 

235 

75.57 

26.08 

66% 

2.90 

The  first  column  denotes  the  number  of  tokens  which  are  parsed.  This  example 
illustrates  that  indexing  can  lead  to  improvements  by  over  a  factor  of  2.90.  In  fact,  the 
more  tokens  need  to  be  parsed  and  the  longer  the  tabled  logic  programming  engine  runs, 
the  larger  the  benefits  of  indexing.  The  table  grows  up  to  over  4000  elements  in  this 
example.  This  indicates  that  indexing  prevents  to  some  extent  program  degradation 
due  to  large  tables  and  longer  run-times. 
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5.4.2  Refinement  type-checker 

In  this  section,  we  discuss  experiments  with  a  bi-directional  type-checking  algorithm 
for  a  small  functional  language  with  intersection  types  which  has  been  developed  by 
Davies  and  Pfenning  [17].  We  use  an  implementation  of  the  bi-directional  type-checker 
in  Elf  by  Pfenning.  The  type-checker  is  executable  with  the  original  logic  programming 
interpreter,  which  performs  a  depth-first  search.  However,  redundant  computation  may 
severely  hamper  its  performance  as  there  are  several  derivations  for  proving  that  a 
program  has  a  specified  type  (see  also  Chapter  4). 

We  give  several  examples  which  are  grouped  in  three  categories.  In  the  first  cat¬ 
egory,  we  are  interested  in  finding  the  first  answer  to  a  type  checking  problem  and 
once  we  have  found  the  answer  execution  stops.  The  second  category  contains  exam¬ 
ple  programs  which  are  not  well-typed  and  the  implemented  type-checker  rejects  these 
programs  as  not  well-typed.  The  third  category  are  examples  where  we  are  interested 
in  hireling  all  answer  to  the  type-checking  problem. 


First  answer 


example 

noindex 

index 

reduction 

time 

improvement 

factor 

subl 

3.19 

0.46 

86% 

6.93 

sub2 

4.22 

0.55 

87% 

7.63 

sub3 

5.87 

0.63 

89% 

9.32 

mult 

7.78 

0.89 

89% 

8.74 

square  1 

9.08 

0.99 

89% 

9.17 

square2 

9.02 

0.98 

89% 

9.20 

Not  provable 


example  noindex  index  reduction  improvement 


time 

factor 

multNPl 

2.38 

0.38 

84% 

6.26 

multNP2 

2.66 

0.51 

81% 

5.22 

plusNPl 

1.02 

0.24 

76% 

4.25 

plusNP2 

6.48 

0.85 

87% 

7.62 

squareNPl 

9.29 

1.09 

88% 

8.52 

squareNP2 

9.26 

1.18 

87% 

7.85 
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All  answers 


example 

noindex 

index 

reduction 

time 

improvement 

factor 

subl 

6.88 

0.71 

90% 

9.69 

sub2 

3.72 

0.48 

87% 

7.75 

sub3 

4.99 

0.59 

88% 

8.46 

mult 

9.06 

0.98 

89% 

9.24 

square  1 

10.37 

1.11 

89% 

9.34 

square2 

10.30 

1.08 

90% 

9.54 

As  the  results  demonstrate  indexing  leads  to  substantial  improvements  by  over  a 
factor  of  9.  Table  sizes  are  around  500  entries. 


5.4.3  Evaluating  Mini-ML  expression  via  reduction 

In  the  third  experiment  we  use  an  implementation  which  evaluates  expressions  of  a 
small  functional  language  via  reduction.  The  reduction  rules  are  highly  non-deterministic 
containing  reflexivity  and  transitivity  rules. 


example  noindex  index  reduction  improvement 


time 

factor 

multi 

10.86 

6.26 

57% 

1.73 

mult  2 

39.13 

18.31 

47% 

2.14 

addminusl 

54.31 

14.42 

73% 

3.77 

addminus2 

57.34 

15.66 

73% 

3.66 

addminus3 

55.23 

25.45 

54% 

2.17 

addminus4 

144.73 

56.63 

61% 

2.55 

minusaddl 

1339.16 

462.83 

65% 

2.89 

As  the  results  demonstrate,  performance  is  improved  by  up  to  3.77.  Table  size  was 
around  500  entries  in  the  table.  The  limiting  factor  in  this  example  is  not  necessarily 
the  table  size  but  the  large  number  of  suspended  goals  which  is  over  6000.  This  may  be 
the  reason  why  the  speed-up  is  not  as  large  as  in  the  refinement  type-checking  example. 
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5.5  Related  work  and  conclusion 

We  have  presented  a  higher-order  term  indexing  technique,  called  higher-order  substitu¬ 
tion  trees.  We  only  know  of  two  other  attempts  to  design  and  implement  a  higher-order 
term  indexing  technique.  L.  Klein  [32]  developed  in  his  master’s  thesis  a  higher-order 
term  indexing  technique  for  simply  typed  terms  where  algorithms  are  based  on  a  frag¬ 
ment  of  Huet’s  higher-order  unification  algorithm,  the  simplification  rules.  Since  the 
most  specific  linear  generalization  of  two  higher-order  terms  does  not  exist  in  general, 
he  suggests  to  maximally  decompose  a  term  into  its  atomic  subterms.  This  approach 
result  in  larger  substitution  trees  and  stores  redundant  substitutions.  In  addition,  he 
does  not  use  explicit  substitutions  leading  to  further  redundancy  in  the  representa¬ 
tion  of  terms.  As  no  linearity  criteria  is  exploited,  the  consistency  checks  need  to  be 
performed  eagerly,  potentially  degrading  the  performance. 

Necula  and  Rahul  briefly  discuss  the  use  of  automata  driven  indexing  for  higher- 
order  terms  in  [42],  Their  approach  is  to  ignore  all  higher-order  features  when  main¬ 
taining  the  index,  and  return  an  imperfect  set  of  candidates.  Then  full  higher-order 
unification  on  the  original  terms  is  used  to  filter  out  the  ones  which  are  in  fact  unifi- 
able  in  a  post-processing  step.  They  also  implemented  Huet’s  unification  algorithm, 
which  is  highly  nondeterministic.  Although  they  have  achieved  substantial  speed-up 
for  their  application  in  proof-carrying  code,  it  is  not  as  general  as  the  technique  we 
have  presented  here.  The  presented  indexing  technique  is  designed  as  a  perfect  filter 
for  linear  higher-order  patterns.  For  objects  which  are  not  linear  higher-order  patterns, 
we  solve  variable  definitions  via  higher-order  unification,  but  avoid  calling  higher-order 
unification  on  the  original  term. 

So  far,  we  have  implemented  and  successfully  used  higher-order  substitution  trees 
in  the  context  of  higher-order  tabled  logic  programming.  The  table  is  a  dynamically 
built  index,  i.e.  when  evaluating  a  query  we  store  intermediate  goals  encountered  dur¬ 
ing  proof  search.  One  interesting  use  of  indexing  is  in  indexing  the  actual  higher-order 
logic  program.  For  this  we  can  build  the  index  statically.  Although  the  general  idea 
of  substitution  trees  is  also  applicable  in  this  setting  there  are  several  important  opti¬ 
mizations.  For  example,  we  can  compute  an  optimal  substitution  tree  via  unification 
factoring  [18]  for  a  static  set  of  terms  to  get  the  best  sharing  among  clause  heads. 
In  the  future,  we  plan  to  adopt  and  optimize  substitution  tree  indexing  for  indexing 
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higher-order  logic  programming  clauses. 
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Chapter  6 

An  implementation  of  tabling 


Tabled  logic  programming  has  been  successfully  applied  in  a  wide  variety  of  applica¬ 
tions,  such  as  parsing,  deductive  databases,  program  analysis  and  more  recently  in 
verification  through  model  checking.  In  previous  chapters,  we  have  shown  that  tabling 
techniques  are  also  very  useful  and  effective  in  the  higher-order  setting.  Although  the 
overall  idea  of  tabling  is  easy  to  understand  on  an  informal  level,  there  is  surprisingly 
little  work  on  how  to  implement  a  tabled  logic  programming  interpreter  efficiently  and 
describing  its  essential  features. 

The  most  successful  implementation  of  tabling  is  found  in  XSB.  It  is  also  the  only 
general  Prolog  system  that  offers  tabling  implemented  at  the  level  of  the  underlying 
abstract  machine.  XSB  is  based  on  the  SLG-WAM,  an  extension  of  the  WAM  (Warren 
Abstract  Machine)  with  special  tabling  support  [62],  Techniques  like  suspending  and 
resuming  computation,  require  a  fair  amount  of  modifications  to  some  of  the  WAM’s 
instructions.  Although  this  approach  seems  to  work  efficiently,  it  is  difficult  to  under¬ 
stand  and  to  re-implement.  The  WAM  itself  already  “resembles  an  intricate  puzzle, 
whose  many  pieces  fit  tightly  together  in  a  miraculous  way”  [5].  The  SLG-Wam  is 
a  carefully  engineered  extension  where  special  support  for  tabling  is  woven  into  the 
WAM  instruction  set.  This  results  in  a  small  run-time  overhead  (roughly  10%),  even 
if  no  tabled  predicate  is  executed.  To  remedy  this  situation  and  provide  a  simpler 
explanation  to  implementing  tabling,  Dernoen  and  Sagonas  [20,  21]  propose  a  different 
approach.  Execution  environments  of  the  WAM  are  preserved  by  copying  the  state  to 
a  separate  memory  area.  In  theory,  this  approach  can  be  arbitrarily  worse,  but  it  is 
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competitive  and  often  faster  than  SLG-WAM  in  practice.  Although  this  approach  does 
not  require  modifications  to  the  WAM  instructions,  it  is  still  very  closely  tied  into  the 
WAM. 

In  this  chapter,  we  describe  the  implementation  of  a  tabled  higher-order  logic  pro¬ 
gramming  interpreter  in  a  functional  language  like  SML.  The  implementation  follows 
closely  the  previous  semi-functional  implementation  of  the  higher-order  logic  program¬ 
ming  interpreter  [23] .  We  describe  a  true  tabled  interpreter,  rather  than  how  to  imple¬ 
ment  tabling  within  logic  programming  or  within  the  WAM.  We  give  an  operational 
interpretation  of  the  uniform  proof  system  presented  in  Chapter  4.  In  particular,  we 
will  focus  on  some  of  the  implementation  issues  such  as  suspending  and  resuming  com¬ 
putation,  retrieving  answers,  and  trailing.  It  is  intended  as  a  high-level  explanation  of 
adding  tabling  to  an  existing  logic  programming  interpreter.  This  hopefully  enables 
rapid  prototype  implementations  of  tabled  logic  programming  interpreters,  even  for  lin¬ 
ear  logic  programming  and  other  higher-order  logic  programming  systems.  In  addition, 
it  facilitates  the  comparisons  and  experiments  with  different  implementations. 


6.1  Background 

We  first  describe  on  a  high-level  how  the  higher-order  logic  programming  interpreter 
works  operationally.  We  omit  a  discussion  on  compilation,  but  rather  focus  on  the 
uniform  proof  system  given  in  Chapter  4.  The  interested  reader  can  find  a  first  step 
on  compiling  uniform  proofs  into  an  intermediate  language  in  [9].  Here  we  give  an 
operational  view  of  the  overall  search  and  discuss  some  of  the  implementation  issues. 

Computation  starts  by  decomposing  the  goal  A,  which  we  are  trying  to  prove,  until 
it  is  atomic. 

1.  Given  an  atomic  goal  P,  look  through  the  program  T  to  establish  a  proof  for  P. 

2.  Given  goal  A{  — >  A2,  add  the  dynamic  assumption  Ai  to  the  programs  in  T  and 
attempt  to  solve  the  goal  A2  from  the  extended  program  T,c:Ai. 

3.  Given  a  universally  quantified  goal  Ylx-.A1.A2.,  we  generate  a  new  parameter  c, 
and  attempt  to  solve  [c/x\A2  in  the  extended  program  context  Y,c.Ai. 
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Once  the  goal  is  atomic,  we  need  to  select  a  clause  from  the  program  context  T  to 
establish  a  proof  for  P.  In  a  logic  program  interpreter,  we  consider  all  the  clauses  in 
T  in  order.  First,  we  will  consider  the  dynamic  assumptions,  and  then  we  will  try  the 
static  program  clauses  one  after  the  other.  Let  us  assume,  we  picked  a  clause  A  from 
the  program  context  T  and  we  now  need  to  establish  a  proof  for  P. 

1.  Given  the  atomic  clause  P' ,  we  establish  a  proof  for  the  atomic  goal  P  by  checking 
if  P'  unifies  with  P .  If  yes  then  succeed.  Otherwise  fail  and  backtrack. 

2.  Given  the  clause  A2  — >  A\,  we  attempt  to  establish  a  proof  of  the  atomic  goal  P, 
by  trying  to  use  the  clause  A\  to  establish  a  proof  for  P.  If  it  succeeds  attempt 
to  solve  the  goal  A2.  If  it  fails,  backtrack. 

3.  Given  the  clause  Yix-.A1.A2,  we  establish  a  proof  for  the  atomic  goal  P  by  gen¬ 
erating  a  new  logic  (existential)  variable  u,  and  use  the  clause  [ufidj/a;]^  to 
establish  a  proof  for  the  atomic  goal  P. 

There  are  several  issues  which  come  up  when  implementing  this  operational  view. 
The  first  question  is  how  to  implement  backtracking.  The  fundamental  idea  underlying 
our  implementation  is  the  concept  of  continuations.  When  solving  a  goal  A2  — »  Ai,  we 
continue  decomposing  the  clause  A\  to  establish  a  proof  for  a  goal  P  and  delay  the  task 
of  solving  the  sub-goal  A2  into  the  success  continuation.  So  we  can  think  of  the  success 
continuation  as  a  stack  of  open  proof  obligations.  Once  we  succeed  in  using  the  clause 
A\  to  establish  a  proof  for  the  goal  P ,  we  call  the  success  continuation.  Backtracking 
is  achieved  by  using  the  internal  failure  continuation  of  the  ML  interpreter.  The  failure 
continuation  is  triggered  by  simply  returning  from  solving  the  goal  Ai  with  an  unin¬ 
teresting  value.  This  is  a  technique  which  goes  back  to  Carlson  [7]  and  is  described  in 
more  detail  in  [23]. 

The  second  implementation  choice  concerns  the  representation  of  variables.  We  use 
de  Bruijn  indices  and  explicit  substitutions  to  implement  bound  variables.  De  Bruijn 
indices  allow  a  standard  representation  without  introducing  names  for  bound  variables. 
As  a  consequence,  we  need  not  re-name  bound  variables.  Explicit  substitutions  [1]  allow 
us  to  delay  applying  substitutions.  In  other  words,  substitutions  are  handled  lazily  in 
the  implementation  and  only  applied  when  needed.  A  similar  idea  has  been  employed 


187 


CHAPTER  6.  AN  IMPLEMENTATION  OF  TABLING 


in  the  Teyjus  compiler  [40]  for  AProlog  seems  crucial  to  efficient  implementations  of 
higher-order  systems.  The  third  choice  concerns  the  handling  of  existential  (logic) 
variables.  We  implement  existential  variables  via  references  and  destructive  updates. 
This  has  several  consequences.  First,  the  existential  variables  occur  free  in  objects.  This 
is  unlike  the  modal  variables  used  in  the  theoretical  development  which  are  declared 
in  a  modal  context  A.  We  can  translate  an  object  U  with  free  existential  variables 
into  a  closed  object  U'  where  existential  variables  are  interpreted  as  modal  variables 
and  declared  in  a  modal  context  A  by  abstracting  over  the  free  existential  variables  in 
U.  The  second  consequence  is  that  whenever  we  instantiate  an  existential  variable  to 
a  term,  any  occurrence  of  this  existential  variable  is  immediately  updated.  Since  these 
updates  continue  to  exist  until  we  undo  them,  we  need  to  make  provisions  to  roll  back 
the  instantiation  of  existential  variables  upon  backtracking.  We  use  a  global  trail  to 
keep  track  of  instantiations  of  existential  variables.  The  trail  is  implemented  as  a  stack 
of  references.  At  a  choice  point,  we  set  a  marker  on  the  trail.  Upon  backtracking  to 
this  choice  point,  we  uninstantiate  all  variables,  which  have  been  pushed  onto  the  trail 
and  simultaneously  unwind  the  trail  (that  is  pop  the  stack).  Therefore,  we  proceed  as 
follows:  when  selecting  a  clause  A  from  the  program  context  T  to  solve  an  atomic  goal 
P ,  we  set  a  choice-point  on  the  trail.  If  we  cannot  establish  a  proof  for  the  goal  P 
using  clause  A,  we  come  back  to  this  choice-point,  unwind  the  trail,  and  try  the  next 
clause  from  T. 

Finally,  we  are  not  only  interested  in  establishing  that  there  exists  a  proof  for  a  goal 
P,  but  in  addition  to  the  answer  substitution,  we  return  the  actual  proof  as  a  proof 
term.  During  proof  search,  we  simultaneously  construct  a  proof  term  M.  This  is  done 
in  the  success  continuation. 

A  more  detailed  description  of  a  semi-functional  implementation  of  a  higher-order 
logic  programming  interpreter  together  with  ML-code  can  be  found  in  [23]. 


6.2  Support  for  tabling 

Next,  we  consider  the  implementation  of  a  tabled  higher-order  logic  interpreter.  In 
Chapter  4,  we  described  the  underlying  idea  and  presented  an  abstract  view  of  a  tabled 
higher-order  logic  programming  interpreter.  Here  we  discuss  the  implementation  of  the 
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tabled  higher-order  logic  programming  interpreter  in  a  semi-functional  language. 


Tabling  eliminates  redundant  and  infinite  computation  by  memoizing  subcomputa¬ 
tions  and  re-using  their  results  later.  When  attempting  to  solve  an  atomic  goal  P  in 
the  program  context  T,  we  first  check  if  the  type  family  that  P  belongs  to  is  marked 
tabled.  If  it  is  not,  then  we  proceed  as  previously.  If  it  is  marked  as  tabled,  then  we 
proceed  as  follows:  We  check  if  we  have  seen  the  atomic  goal  P  in  context  T  before. 
If  the  current  atomic  goal  P  in  context  T  is  not  in  the  table,  we  add  it  and  proceed. 
Once  we  have  established  a  proof  for  it,  we  will  add  the  answer  to  it  in  the  table. 


If  the  atomic  goal  P  in  context  T  is  already  in  the  table,  then  there  are  two  choices: 
If  there  are  answers  available,  we  would  like  to  re-use  them  and  continue.  If  no  answers 
are  available,  then  we  need  to  suspend  the  execution  and  resume  computation  later 
on.  In  this  case,  we  copy  the  trail  together  with  the  atomic  goal  P  and  the  program 
context  T  and  execution  just  fails.  The  overall  search  backtracks  to  the  last  choice 
point. 


To  illustrate  the  search  process,  recall  the  example  given  in  Chapter  4.  We  first 
repeat  the  two  clauses  of  the  implementation  of  the  type  system  including  subtyping. 


tp_sub 

:  of  ET’ 

tp_lam  : 

of  (lam  ( [x]  Ex))  (T1  => 

T2) 

<-  of  E  R 

<-  ({x:exp}  of  x  T1  ->  of 

(E  x)  T2) 

<-  sub  R  T’. 

Next,  recall  the  proof  tree  for  computing  the  types  of  the  identity  function. 
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Stage  1 


of  (lam  [x]  x)  T  (T) 

JPjyb,  of  (lam  [x]  x)  R,  @  Suspend 
sub  R  T 


tpJam  -  u:ofxT1h  of  x  T2 

© 


T1  =  T3,  T2  =  T3,  T  =  T3  => 

u:of  x  T1  h  of  x  R,  (5)  < 

sub  R  T2 


We  start  with  the  goal  of  (lam  [x]  x)  T  marked  with  number  (1).  Before  we  apply 
the  program  clause  tp_sub,  we  put  a  choice-point  marker  on  the  trail.  Once  the  head  of 
E  T'  of  the  clause  tp_sub  unifies  with  the  current  goal,  we  add  to  the  trail  T’  :=  T.  Since 
the  goal  of  (lam  [x]  x)  R  is  a  variant  of  the  original  goal,  computation  is  suspended,  and 
the  overall  search  fails  and  backtracks  to  the  last  choice  point.  Then  we  try  the  next 
clause  tpJam.  When  we  unify  the  head  of  (lam  [x]  E  x)  (T1  =>  T2)  with  the  current 
goal,  we  add  two  more  entries  to  the  trail.  The  evolution  of  the  trail  up  to  subgoal  (4) 
is  depicted  below.  Each  column  is  labelled  with  the  subgoal  it  belongs  to. 
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T2  :=  T3 

T1  :=  T3 

mark 

T  :=  T1  ^  T2 

T  :=  T1  ^  T2 

T  :=  T1  =►  T2 

T’  :=  T 

m 

'll 

2 

X 

m 

'll 

2 

X 

E  :=  [x]  x 

mark 

mark 

mark 

mark 

mark 

mark 

(1)  (2)  (1)  (3)  (4)  (3) 

Next,  we  will  discuss  the  table,  answers  and  suspended  goals. 


6.2.1  Table 

The  table  plays  a  central  role  in  the  overall  design  of  the  tabled  logic  programming 
interpreter.  Recall  the  definition  of  a  table  given  earlier  in  Chapter  4.  A  table  T  is 
a  collection  of  table  entries.  A  table  entry  consists  of  two  parts:  a  goal  A;  T  -A*  a 
and  a  list  A  of  pairs,  answer  substitutions  A'  b  6  :  A  and  proof  terms  M,  such  that 
A';  [#]r  —A  \6\M  :  [#]a  is  a  solution. 

A  central  question  is  then  how  to  implement  the  table,  store  the  table  entries  and 
answers,  and  define  the  right  interface  for  accessing  the  table.  We  start  with  some 
general  remark  about  the  table  and  its  entries.  First  of  all,  we  implement  for  each 
tabled  predicate  a  separate  table  and  each  table  will  be  stored  in  a  reference  cell.  So 
we  have  an  array  where  the  element  i  of  the  array  contains  the  pointer  (or  reference) 
to  the  table  of  predicate  i. 

Second,  we  need  to  ensure  quick  table  access.  As  we  have  shown  in  Chapter  3, 
higher-order  pattern  unification  may  not  be  efficient  in  general,  but  we  can  obtain  an 
efficient  algorithm  for  linear  higher-order  patterns.  To  ensure  quick  table  access  we 
therefore  linearize  the  atomic  goal  P  and  the  context  T.  As  a  result,  we  obtain  a  linear 
atomic  goal  a;  and  a  linear  context  together  with  some  variable  definitions  e. 

Third,  table  entries  must  be  closed  in  the  sense  that  they  are  not  allowed  to  con¬ 
tain  any  references  to  existential  variables.  References  to  existential  variables  may  be 
instantiated  during  proof  search  which  pollutes  the  table  and  would  change  the  table 
entries  destructively.  To  avoid  this  problem,  we  de-reference  and  abstract  over  all  the 
existential  variables  in  a  term  before  inserting  it  into  the  table.  We  translate  a  term 
where  existential  variables  may  occur  free  into  a  term  which  is  closed  and  existential 
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variables  are  translated  into  modal  variables  which  are  bound  in  the  modal  context  A. 
Abstraction  of  the  atomic  goal  P  in  program  context  T  returns  four  parts: 

•  A:  is  the  context  for  the  modal  variables 

•  r':  is  the  abstraction  of  the  context  T 

•  P'\  is  the  abstraction  of  the  atomic  goal  P. 

•  e'\  is  the  abstraction  of  the  variable  definitions  e. 

By  invariant  the  abstracted  atomic  goal  P'  is  well-typed  in  the  ordinary  context  T' 
and  the  modal  context  A  modulo  the  variable  definitions  e' .  A  table  entry  consists  of 
this  abstraction  together  with  a  reference  to  the  corresponding  answer  list. 


modal  context 

dynamic  context 

atomic  goal 

variable  definitions 

Answer  Ptr 

A 

r 

P' 

e 

answRef 

Table  Entry 

Now  we  can  check  if  the  abstracted  linear  atomic  goal  P'  together  with  the  context 
T,  the  modal  context  A  and  the  variable  definitions  e  is  already  in  the  table.  To  be 
more  precise,  if  there  is  a  table  entry  which  is  a  variant  of  the  abstracted  goal.  Since 
we  use  de  Bruijn  indices  for  representing  bound  variables,  and  all  entries  are  in  normal 
form,  this  can  be  done  by  checking  syntactic  equality.  If  the  result  is  yes,  we  will  return 
a  pointer  to  the  answer  list.  If  the  result  is  no,  we  add  it  to  the  table  together  with  a 
pointer  to  an  empty  answer  list  and  also  return  a  pointer  answRef  to  the  still  empty 
answer  list.  The  intention  is  that  once  the  goal  P  in  context  T  has  been  solved,  we  add 
the  answer  substitution  to  the  answer  list  answRef  is  pointing  to.  This  is  done  in  the 
success  continuation. 

This  task  will  be  just  added  to  the  success  continuation.  In  other  words  the  success 
continuation  is  not  only  a  stack  of  open  proof  obligations  (subgoals),  but  also  of  other 
obligations  such  as  adding  answers  to  the  table.  We  illustrate  this  idea  again  by 
considering  the  previous  example.  At  node  (5),  the  success  continuation  looks  as 
follows. 
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solve  (sub  R  T2)  using  the  program  clauses  and  the  dynamic  assumption  of  x  T 

add  answers  for  T1  and  T2  to  the  answRef  corresponding  to  of  x  T1  b  of  x  T2 

add  answer  for  T  to  answRef  corresponding  to  of  (lam  [x]  x)  T 

initial  success  continuation 

It  not  only  contains  the  open  subgoal  sub  R  T2,  but  also  the  tasks  of  adding  answers. 
It  is  important  to  note  that  adding  answers  to  a  table  entry  does  not  require  another 
table  lookup,  since  we  have  direct  access  to  the  reference  of  the  corresponding  answer 
list. 

To  facilitate  adding  answer  substitutions,  we  extend  the  abstraction  process  slightly. 
Given  a  linear  atomic  goal  cq  in  a  context  Id  and  the  variable  definitions  e  let  Ad, ... ,  Xn 
be  the  free  existential  variables  occurring  in  it.  We  not  only  compute  the  abstraction 
A;  T'  b  a'  together  with  the  variable  definitions  e',  but  also  return  a  substitution  6  from 
the  context  A  to  the  existential  variables  Ad, ... ,  Xn.  In  other  words,  the  substitution 
6 ,  maps  every  abstracted  existential  variable  in  A  to  its  reference  such  that  [djT'  b  [d]a/ 
modulo  the  variable  definitions  [ 6\e'  is  a  variant  of  the  original  the  original  goal  T  b  a. 

Creating  such  a  substitution  6  for  the  existential  variables  is  important  for  mainly 
two  reasons:  (1)  As  we  will  establish  a  proof  for  the  goal  T  b  a,  its  existential  variables 
will  be  instantiated.  As  a  side-effect,  6  will  exactly  contain  these  instantiations.  Once 
we  have  found  a  proof  for  T  b  a,  9  will  contain  the  answer  substitution,  and  we  can 
directly  add  9  to  the  answer  list,  without  another  additional  table  lookup.  (2)  Any 
retrieval  operations  will  require  no  additional  table  lookup  up.  Since  the  substitution 
9  captures  the  free  variables  in  the  goal,  and  we  have  direct  access  to  the  answer  list, 
subsequent  retrieval  operations  do  not  again  traverse  the  index  together  with  the  term, 
but  can  work  solely  on  the  substitutions.  Therefore,  we  do  exactly  one  lookup  in  the 
table  for  each  tabled  goal. 

6.2.2  Indexing  for  table  entries 

The  table  itself  is  implemented  via  substitution  tree  indexing.  We  index  on  the  ab¬ 
stracted  linear  atomic  goal.  At  the  leafs  of  the  substitution  tree  we  not  only  store  the 
variable  definitions  e',  but  also  the  context  T  and  the  modal  context  A.  In  addition, 
we  keep  a  pointer  to  the  corresponding  answer  list  at  the  leaf  (see  Figure  6.2.2). 
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There  have  been  several  reasons  for  this  design,  which  we  briefly  discuss.  First, 
indexing  on  the  goal,  rather  than  on  the  context  A  and  T,  allows  for  more  structure 
sharing,  since  often  the  goals  are  similar,  but  the  context  T  and  A  may  differ.  In 
addition  it  allows  direct  access  to  the  goal  of  the  table  entry  and  allows  to  implement 
a  table  for  each  type  family.  Finally,  the  design  is  suitable  for  other  optimization  such 
as  strengthening  or  context  subsumption. 

6.2.3  Answers 

Next,  we  will  describe  adding  answers  and  the  answer  list  itself  in  more  detail.  Once  we 
have  solved  a  tabled  goal,  we  need  to  add  the  answer  substitution  9  to  the  answer  list. 
Recall  that  checking  whether  the  table  entry  (A;  T';  a';  e )  is  already  in  the  table,  returns 
a  reference  answRef  to  an  answer  list.  Once  we  have  established  a  proof  for  this  entry, 
we  need  to  add  the  corresponding  answer  substitutions  9  to  answRef.  When  adding 
the  answer  substitution  to  the  answer  list,  there  are  two  important  observations  to  take 
into  account.  First,  answer  substitutions  9  may  contain  free  existential  variables,  as  the 
previous  example  illustrates.  Again  we  need  to  ensure  that  any  answer  substitutions 
in  the  table  are  ’’closed”  to  avoid  pollution  of  the  table.  Therefore  we  abstract  over 
the  answer  substitution.  Let  9’  together  with  the  modal  context  A  be  the  abstraction 
of  the  answer  substitution  9. 

The  second  observation  is  that  we  not  only  generate  answer  substitutions,  but  also 
proof  terms.  One  way  to  handle  this  issue  is  to  store  not  only  the  abstracted  answer 
substitution  O'  together  with  the  modal  context  A,  but  also  the  proof  term.  In  the 
previous  example,  the  corresponding  proof  term  to  node  (3)  is  (tpJam  [w:of  x  P]  u)1. 
In  practice  however  the  corresponding  proof  terms  may  be  large  and  may  lead  to  severe 
performance  penalties.  Since  the  size  of  the  actual  proof  term  may  be  large,  it  would 
not  be  very  space  efficient  to  store  them.  Second,  we  also  would  need  to  compute 
proof  terms  simultaneously  during  proof  search.  This  also  may  be  expensive.  In  the 
implementation,  we  therefore  choose  to  only  carry  around  a  skeleton  of  the  proof  term, 
from  which  the  actual  proof  can  be  reconstructed.  The  skeleton  keeps  track  of  clauses 
have  been  applied  to  establish  a  proof,  without  keeping  track  of  typing  dependencies. 
For  static  clauses  we  take  the  identifier  of  the  clause  name  and  for  dynamic  clauses  we 

1We  omitted  here  the  implicit  argument  x.  The  explicit  form  is  (tpJam  [x:exp]  [u:of  x  P\  it). 
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take  their  position  in  the  dynamic  context  T.  So  in  the  previous  example,  we  store  the 
identifier  of  tpJam  and  a  1,  denoting  we  used  the  first  dynamic  assumption.  This  is 
a  simplified  version  of  Necula.  and  Rahul’s  proposal  of  oracles  [42],  Necula.  and  Rahul 
are  even  more  minimalistic  since  they  only  encode  the  non-deterministic  choices  in  the 
proof.  In  our  setting,  the  answer  list  is  set  up  as  follows: 


modal  context 

answer  substitution 

skeleton 

A' 

9' 

sk 

Answer  List 

Finally,  we  need  to  have  some  more  control  on  what  answer  substitutions  are  allowed 
to  be  re-used.  As  described  previously,  we  enforce  a  multi-stage  strategy,  where  we  are 
only  allowed  to  use  answers  from  previous  stages.  Hence  we  also  add  a  lookup  counter 
n,  which  indicates  any  element  greater  then  n  is  not  allowed  to  be  used. 

The  access  cost  for  answers  is  proportional  to  the  size  of  the  answer  substitution, 
rather  than  to  the  size  of  the  answer  itself.  This  idea  is  also  known  as  substitution 
factoring  [59].  As  we  see,  this  idea  can  be  adopted  to  the  higher-order  setting  in  a 
straightforward  way  and  modal  substitutions  inherently  support  substitution  factoring. 
Every  answer  is  essentially  a  reference  to  the  actual  answer  record  which  allows  direct 
access  to  answer  substitutions. 

To  achieve  a  more  efficient  representation  of  the  solutions,  it  is  possible  to  replace 
the  list  of  pairs,  answer  substitutions  and  proof  skeletons  with  a  trie.  This  would 
allow  sharing  of  prefixes  of  substitutions.  However,  since  in  most  example  we  did  not 
encounter  large  lists  of  answer  substitutions,  we  have  not  pursued  this  optimization. 

6.2.4  Suspended  goals 

If  we  detect  a  a  goal  which  is  a  variant  of  a  table  entry,  we  need  to  suspend  the 
computation.  We  only  need  to  store  enough  information  to  recover  and  retrieve  answers 
once  they  become  available.  In  fact,  we  only  need  to  store  a  substitution  9,  which 
contains  the  free  existential  variables,  such  that  •  b  6  :  A.  In  addition,  we  store 
the  success  continuation  sc  and  the  pointer  answRef  to  the  answer  list.  The  idea  is 
that  once  answers  become  available,  we  instantiate  the  substitution  9  with  the  answer 


195 


CHAPTER  6.  AN  IMPLEMENTATION  OF  TABLING 


substitution  from  answRef .  As  we  have  direct  access  to  answRef  and  we  need  no 
additional  table  lookup,  we  do  not  need  to  store  the  complete  goal.  We  also  do  not 
store  the  stack  of  failure  continuations.  When  resuming  the  suspended  goals,  we  will 
not  backtrack  past  them,  since  any  earlier  choice  points  have  already  been  explored. 
A  similar  observation  has  been  made  by  Demoen  and  Sagonas  [19].  This  also  justifies 
our  choice  to  re-use  the  ML  internal  continuation  as  a  failure  continuation,  even  if  we 
add  tabling  to  the  interpreter. 

This  is  however  still  not  quite  sufficient.  The  problem  stems  again  from  the  existen¬ 
tial  variables  which  are  implemented  via  references.  Once  we  come  back  and  resume 
the  computation,  the  free  existential  variables  in  the  substitution  6  and  in  the  success 
continuation  may  not  be  in  the  same  state  anymore.  Therefore,  we  need  to  be  able  to 
capture  the  state  of  the  existential  variables  and  reconstruct  it.  We  do  this  by  copying 
the  trail  and  storing  a  copy  of  the  trail  together  with  the  substitution  9,  the  success 
continuation  and  the  pointer  to  the  answer  list. 

When  copying  the  trail,  we  can  remove  all  the  marks  from  the  trail,  since  we  do  not 
need  to  keep  the  choice  points.  In  other  words,  we  only  need  to  store  pairs  of  references 
and  terms. 


ref 

Trail  Copied  Trail 

Copying  the  trail  may  be  quite  expensive  and  it  may  be  possible  to  share  common 
prefixes  of  the  trails  in  suspended  goals  or  even  with  the  current  trail.  However,  we 
have  not  investigated  this  idea  further. 

Finally,  we  need  some  more  control  on  which  answers  from  the  answer  list  have  been 
already  used.  Therefore,  we  associate  with  every  suspended  goal  a  lookup  pointer  k  to 
which  answers  have  already  been  used.  So  the  suspended  goals  consist  of  the  following 
5  parts. 


T’ 

T1 

T 

T1  =>  T2 

E 

M  x 

ref.  inst. 


T’  :=  T1 

mark 

T  :=  T1  =*►  T2 
mark 
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substitution 

success  cont. 

copied  trail 

lookup  ptr 

reference  to  answer 

e 

sc 

ctra.il 

k 

answRef 

Suspended  Goal 

All  the  suspended  goals  are  stored  in  a  global  queue,  and  resumed  in  the  order  they 
were  suspended. 

6.2.5  Resuming  computation 

To  resume  suspended  goals,  we  set  a  choice-point  and  reset  the  existential  variables  on 
the  copied  trail  to  their  previous  state  when  we  suspended  the  computation.  Then  we 
resume  the  computation  by  retrieving  answers  from  the  answer  list  answRef . 

Recall  that  we  use  a  multi-stage  strategy,  which  restricts  the  the  re-use  of  answers 
to  the  ones  which  have  been  derived  in  previous  stages.  Consider  the  following  answer 
list  to  illustrate. 


(Ai,0i),  ski 

(A2,02),  sk2 

(A3,03),  sk3 

(A 4,6*4),  sk4 

k 

n 

n  marks  the  answers  generated  in  previous  stages  and  any  answers  smaller  than  n 
are  allowed  to  be  re-used.  The  index  k  associated  with  every  suspended  goal,  indicates 
that  all  answers  smaller  than  k  have  already  been  re-used.  Hence,  we  only  retrieve 
answers  between  k  and  n. 

Retrieval  of  an  answer  substitution  is  done  in  two  steps:  First,  we  create  new 
existential  variables  for  each  element  in  Aj,  s.t.  p  is  a  substitution  with  these  fresh 
existential  variables  from  modal  context  A*  to  the  empty  context.  The  composition 
\p\9i  then  re-introduces  fresh  existential  variables  to  the  closed  answer  substitution 
6i .  Second,  we  unify  \p\6i  with  the  substitution  6.  If  this  succeeds,  we  return  the 
corresponding  proof  skeleton  skt  to  the  success  continuation  and  proceed  to  solve  the 
success  continuation. 

If  re-using  the  answer  substitution  0t  failed,  we  backtrack  the  the  last  choice-point, 
and  reuse  the  next  answer  substitution  Qi+\.  If  all  answer  substitutions  have  been 
re-used,  we  backtrack  and  try  another  suspended  goal. 
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6.2.6  Multi-staged  tabled  search 

Tabled  search  proceeds  in  stages.  A  stage  terminates  if  all  the  nodes  in  the  proof  search 
tree  are  either  success  nodes  or  failure  nodes.  Once  a  stage  terminates,  we  check  if  the 
table  has  changed,  i.e.  new  answers  have  been  added.  If  this  is  the  case,  then  we  revive 
the  suspended  goals  and  resume  computation  with  the  newly  derived  answers.  If  no 
new  answers  and  no  new  table  entries  have  been  added,  i.e.  the  table  did  not  change, 
we  are  done  and  the  search  space  has  been  saturated. 

At  the  end  of  each  stage,  we  update  the  answer  pointer  n,  which  indicates  which 
answers  were  generated  in  previous  stages  and  are  therefore  allowed  to  be  reused. 


6.3  Variant  and  subsumption  checking 

So  far  we  have  described  tabled  higher-order  logic  programming  based  on  variant  check¬ 
ing.  The  implementation  is  carefully  designed  and  makes  use  of  some  important  invari¬ 
ants  to  achieve  efficient  execution.  For  example,  for  each  tabled  goal  we  do  exactly  one 
lookup  in  the  table.  By  returning  the  pointer  to  the  answer  list,  subsequent  retrieval 
operations  only  access  the  answer  list,  but  do  not  again  traverse  the  index  together 
with  the  term.  Moreover,  no  overhead  is  imposed  for  non-tabled  evaluation,  except  one 
check  whether  the  predicate  is  declared  tabled  or  not.  Any  changes  needed  for  tabling 
do  not  affect  the  execution  of  non-tabled  predicates. 

Subsumption  checking  allows  us  to  check  whether  there  is  a  more  general  entry 
already  in  the  table  such  that  the  current  goal  is  an  instance  of  it.  A  clear  advantage 
of  subsumption  checking  is  that  it  can  potentially  detect  more  loops  and  leads  to 
smaller  table  sizes.  Unfortunately,  we  cannot  maintain  all  the  invariants  we  exploit 
for  variant-based  lookups.  In  particular,  there  may  be  more  than  one  entry  which  is 
more  general  than  the  current  goal.  Therefore  a  complete  subsumption-based  strategy 
needs  to  return  all  answer  lists  of  these  entries.  This  means  that  for  each  tabled  goal, 
we  may  need  more  than  one  table  lookup.  When  we  encounter  a  tabled  goal,  we  need 
to  check  whether  it  is  already  in  the  table.  Later,  any  time  when  we  retrieve  answers 
for  it,  we  need  to  perform  another  table-look  up. 

Experience  in  XSB  has  shown  that  although  the  table  size  may  be  smaller,  the 
additional  overhead  required  for  subsumption  often  does  not  lead  to  performance  im- 
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provements  in  tabled  logic  programming. 

In  the  current  implementation  of  the  tabled  higher-order  logic  programming  in¬ 
terpreter,  we  have  tried  to  minimize  the  changes  and  preserve  the  design  for  variant¬ 
checking.  In  particular,  when  a  goal  is  added  to  the  table,  we  still  return  a  pointer  to 
its  answer  list,  and  we  will  retrieve  answers  only  from  this  list.  This  may  possibly  delay 
the  reuse  of  answers  and  lead  to  incompleteness,  but  it  avoids  the  effort  of  maintaining 
two  different  implementations. 

The  design  previously  discussed  is  then  extended  to  allow  subsumption  checking 
in  the  following  way:  Look-up  in  the  table  is  subsumption  based,  i.e.  we  check  if  the 
current  abstracted  linear  atomic  goal  P'  in  context  T'  and  modal  context  A'  is  an 
instance  of  a  table  entry  A;  T  b  P.  Recall  that  all  table  entries  are  in  canonical  form. 
Therefore,  A';  T'  b  P'  is  an  instance  of  A;  T  b  P,  if  there  exists  a  substitution  p  such 
that  A'  b  p  :  A  and  A;  [p]T  b  [p]P  •<===>■  P' .  In  this  case,  we  return  the  substitution 
p  together  with  a  reference  answRef  to  the  associated  answer  list.  If  A';T;  b  P'  is  a 
variant  of  the  table  entry  A;  T  b  P  then  p  will  be  a  renaming  substitution. 

For  retrieval,  we  then  unify  the  composition  of  p  and  a  with  the  re-instantiated 
answer  substitution  dj.  This  is  a  conservative  approach,  which  ties  each  goal  to  a  unique 
table  entry  and  avoids  another  table  lookup,  once  we  retrieve  answers.  Alternatively, 
we  could  do  another  table  lookup  when  retrieving  answers.  We  retrieve  all  answer  lists 
of  table  entries  such  that  the  current  goal  is  an  instance  of  the  entry.  Note  that  in 
general  there  may  be  more  than  one  such  table  entry.  This  may  allow  us  to  retrieve 
answers  earlier,  because  we  are  not  tied  to  one  table  entry.  However,  it  requires  an 
additional  lookup  every  time  we  retrieve  answers. 

Although  subsumption-checking  allows  many  more  optimizations  and  may  lead  to 
smaller  tables,  we  have  not  found  a  substantial  performance  improvement.  This  is  in 
fact  consistent  with  observations  made  in  the  XSB  system  with  subsumption-based 
tabling.  On  the  other  hand,  theorem  provers  always  rely  on  subsumption  rather  than 
variant  checking  to  eliminate  redundancy.  As  we  will  discuss  in  the  next  Chapter, 
subsumption-based  checking  may  be  more  useful  in  higher-order  theorem  proving. 
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Chapter  7 

Higher-order  theorem  proving  with 
tabling 


In  this  chapter,  we  concentrate  on  automating  proof  search  in  Twelf.  Twelf  not  only 
provides  a  higher-order  logic  programming  interpreter,  but  also  a  meta-theorem  prover 
for  reasoning  with  the  inference  rules  of  a  deductive  system  and  also  about  deduc¬ 
tive  systems  using  induction.  The  meta-theorem  prover  supports  three  key  operations, 
splitting  a  proof  into  different  cases,  generating  instances  of  the  induction  hypothe¬ 
sis  and  deriving  a  conjecture  from  a  given  set  of  assumptions  and  rules  of  inference. 
Using  these  three  key  operations,  Twelf  can  prove  many  theorems  and  lemmas  about 
specifications  automatically. 

Currently,  the  general  proof  search  strategy  used  in  Twelf  is  based  on  iterative 
deepening  proof  search.  Although  this  works  in  many  examples,  it  also  has  clear 
limitations.  First,  as  the  user  has  to  specify  a  bound,  failure  of  proof  search  becomes 
meaningless.  The  user  does  not  know  whether  the  bound  was  too  low  or  whether 
there  exists  no  proof  for  the  conjecture.  Moreover,  often  the  performance  of  the  search 
procedure  highly  depends  on  choosing  the  minimal  bound.  Although  in  theory  it 
should  not  matter,  if  we  give  the  minimal  bound,  in  practice  this  may  often  be  crucial 
for  finding  the  proof. 

Second,  redundant  computation  may  severely  hamper  the  performance  of  proof 
search  in  Twelf  in  larger  and  more  complex  specifications.  To  prove  properties  about 
deductive  systems  often  requires  several  lemmas  and  their  application  may  lead  to  an 
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explosion  in  the  search  space.  Iterative  deepening  search  does  not  scale  well  to  larger 
examples  and  often  fails  to  prove  key  lemmas  and  theorems  due  the  large  search  space. 
The  experience  with  memoization  in  logic  programming  in  the  other  hand  suggests  that 
memoization  in  theorem  proving  may  be  useful  to  eliminate  redundancy  and  artificial 
depth  bounds  on  the  proof  search  in  many  cases.  In  this  chapter,  we  present  a  generic 
meta-theorem  prover  based  on  memoization  and  discuss  several  examples  including 
type  preservation  proofs  for  type-system  with  subtyping,  decomposition  lemmas  for 
a  reduction  semantics  using  evaluation  context,  several  inversion  lemmas  about  re¬ 
finement  types,  and  reasoning  in  classical  natural  deduction  calculus.  These  examples 
include  several  lemmas  and  theorems  which  were  not  provable  with  iterative  deepening, 
but  can  be  proven  with  memoization-based  search.  Moreover,  we  show  that  in  some 
cases  no  bound  is  needed  on  memoization-based  search.  This  allows  us  to  disprove 
some  sub-cases  in  inductive  proofs  and  show  that  no  proof  exists  to  the  theorem.  This 
is  an  important  step  towards  a  more  robust  and  more  efficient  meta-theorem  prover. 

7.1  Example:  type  system  with  subtyping 

As  a  motivating  example,  we  discuss  a  type  system  with  subtyping  which  we  is  a 
variant  of  the  system  already  introduced  in  Chapter  4.  In  particular,  we  will  present 
some  proofs  relating  the  declarative  subtyping  relation  to  a  deterministic  subtyping 
algorithm  and  show  that  the  deterministic  subtyping  algorithm  is  sound  and  complete. 
Finally  we  show  that  type  preservation  holds. 

To  make  the  chapter  self-contained,  we  start  with  defining  the  types.  Included  are 
the  base  types  nat  for  natural  numbers  which  includes  the  positive  numbers  and  zero, 
zero  denotes  the  number  zero  and  pos  contains  all  positive  numbers.  Numbers  are 
constructed  by  z  and  the  successor  function  s  . 

Expressions  M  ::=  z  |  s  M  |  lam  x.M  |  app  M{  M2  \  pair  Mi  M2  \ 

fst  M  |  snd  M  |  case  M  of  z  M0  j  (s  x)  M\ 

Types  A  ::=  nat  |  zero  |  pos  |  A\  — »  A2 

The  subtyping  judgment  for  this  language  has  the  form  : 

A  <  B  Type  A  is  a  subtype  of  B 

The  subtyping  relation  is  defined  straightforwardly  using  reflexivity  and  transitivity 
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by  the  following  rules 


A  <A 


Ai  <  A 2  A2  A  A$ 

Ai  A  A3 


pos  <  nat  zero  <  nat 

Ai  A  B\  A2  A  i?2  B\  <  Ai  A2  <  i?2 

*  .T2  ^  B\  >k  B2  Ai  — >  A2  A  B\  — >  B2 

Next,  we  give  the  typing  judgment  and  the  evaluation  judgment: 

T  b  M  :  A  Term  M  has  type  A  in  context  T 
M  V  Term  M  evaluates  to  value  V 


Typing  rules 

T  b  M  :  A'  A'  <  A 
T  h  M:  A 

_  _  Thtf :  nat  ThM:  nat 

Thz:  nat  Thz  :  zero  ThsM  :  nat  Tbs  M  :  pos 

ThM:  nat  T  b  M\  :  A  T,  x:nat  b  M2  :  A 
T  b  case  M  of  z  =>  Mi  |  (s  x)  =>•  M2  :  A 

T  b  M  :  zero  TbM,  :  bl  TbM:  pos  T,  x:nat  b  M2  :  A 

T  b  case  M  of  z  4  Mx  |  (s  x)  =t  M2  :  A  T  b  case  M  of  z  4-  Mi  |  (s  x)  =t  M2  :  A 

T  b  tf !  :  T  b  M2  :  bl2  TbM:  (A  *  bL2)  TbM:  (Ai  *  bL2) 

T  b  pair  Mi  M2  :  (b^  *  bf2)  T  b  fst  M  :  Ax  T  b  snd  M  :  A2 

r,  x:Ai  b  M  :  A2  T  b  Mx  :  (A2  ->  A)  TbM2:i2 

T  b  lam  x.M  :  (Ai  — »  T2)  T  b  app  Mx  M2  :  A 

M^V 

zT^Tz  s  M  ^  s  V 
Evaluation  rules 

M  ^  M'  N  ^  N'  M  pair  Vj E2  M  ^  pair  Vj lb 
pair  M  N  ^  pair  M'  TV'  fst  M  ^  fh  snd  M  ^  V2 
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_ M  ^  z  Mi  ^  VI _  M  ^  (s  V2)  [V2/x\M2  ^  V2 

case  M  of  z  4-  Mi  |  (s  x)  =$■  M2  Vi  case  M  of  Mi  |  (s  x)  =>•  M2  V2 

Mi  ^  lam  x.M'  M2  ^  V2  [V2/x]M‘ 
lam  x.M  lam  x.M  app  Mi  M2  V 

The  following  substitution  principle  holds: 

Lemma  87  (Substitution  principle) 

If  T,  x:A  h  M  :  B  and  Th  N  :A  then  T  h  [A/x]M  :  5. 

Proof:  Induction  on  the  first  derivation.  □ 

The  subtyping  rules  above  do  not  immediately  yield  an  algorithm  for  deciding 
subtyping.  Thus  we  will  present  an  algorithmic  subtyping  judgment  and  show  that  it 
is  equivalent  to  the  one  above. 

A  <  B  Type  A  is  a  subtype  of  B 

nat  <  nat  pos  <j  pos  zero  <  zero 

zero  <  nat  pos  <j  nat 

Ai  <B1  A2<  B2  Bi  <Ai  A2<  B2 

[Ai  *  A2)  <|  (i?i  *  B2 )  [Ai  — >  A2)  <|  (i?i  — ■>  B2 ) 

Theorem  88  (Admissibility  of  reflexivity) 

For  any  type  A,  there  exists  a  derivation  A  <  A. 

Proof:  Structural  induction  on  A.  □ 

Theorem  89  (Admissibility  of  transitivity) 

If  T>i  :  A<  B  and  T>2  :  B  <j  C  then  £  :  A<C. 
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Proof:  The  size  of  a  judgment  A  <  B  is  determined  by  the  size  of  A  plus  the  size  of 
B.  Induction  on  the  size  of  the  judgment1.  □ 

Theorem  90  (Soundness  and  completeness  of  algorithmic  subtyping) 

1.  If  8  :  A\  <  A2  then  V  :  A\  <  A2 

2.  If  V  :  A 1  <  A2  then  8  :  A\  <  A2 

Proof:  Soundness:  Proof  by  structural  induction  on  8. 

Completeness:  Proof  by  structural  induction  on  T>  referring  to  reflexivity  and  transi¬ 
tivity  lemma.  □ 

To  prove  type  preservation,  we  need  a  couple  of  inversion  lemmas. 

Theorem  91  (Inversion) 

1.  If  T  b  z  :  A  and  A  <  pos  then  this  is  a  contradiction. 

2.  If  T  b  s  M  :  A  and  A  <  pos  then  T  b  M  :  nat. 

3.  If  r  b  s  M  :  A  and  A  <  nat  then  T  b  M  :  nat. 

4 ■  If  T  P  s  M  :  A  and  A  <j  zero  then  this  is  a  contradiction. 

5.  If  T  b  lam  x.M  :  A  and  A  <  {A\  — >•  A2)  then  T,  x\A\  h  M  :  A2. 

6.  If  r  P  pair  M\  M2  :  A  and  A  <  (A\  *  A2)  then  P  b  Mi  :  A2  and  T  b  M2  :  A2. 

Proof:  All  the  proofs  follow  by  structural  induction  on  the  first  derivation.  □ 

Theorem  92  (Type  preservation) 

If  V  :  r  b  M  :  A  and  8:  M^V  then  T  b  V  :  A. 

1  Because  of  the  contra-variance  of  types  in  the  subtyping  rule  for  function  types,  the  induction 
ordering  is  a  little  tricky.  In  Tuielf,  we  use  a  trick  to  handle  this  induction.  We  use  simultaneous 
induction  on  D 1  and  D2  and  define  a  second  copy  of  the  transitivity  relation  to  handle  the  contra- 
variance  case,  by  a  structural  nested  induction  on  V\  and  V 2. 
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Proof:  Structural  lexicographic  induction  on  T>  and  £  using  the  previously  proven 
lemmas.  We  only  show  a  few  cases  here  to  illustrate  the  overall  structure  of  the  proof. 


D  i  T>2 

Case  V  =  r  \~B<A 

r  b  M  :  A 


M  ^  V 
T  b  V  :  B 
r  b  V  :  A 


Case  V 


v1  v2 

r  b  Mi  :  (A  ->  B)  r  h  M2:A 

T  h  (app  Mi  M2)  :  B 


by  assumption 
by  induction  hypothesis 

by  tp_sub 


app  Mi  M2  >  V  by  assumption 

Mi  (A x.M')  and  M2  V2  and  [ V2 / x]  M'  V  by  inversion 


rhV2:  A 

r  h  A  x.M'  :(A^B) 
r,  x:A  h  M'  :  B 
r  b  [V2/x\M'  :  B 
fhb:B 


by  induction  hypothesis 
by  induction  hypothesis 
by  inversion  lemma 
by  substitution  lemma 
by  induction  hypothesis 


Case  V  = 

r  h  (fst  M)  :  A 


fst  M  ^  Vi 

by  assumption 

M  ^  (pair  Vi  V2) 

by  inversion 

r  h  (pair  Id  V2)  :(A*B ) 

by  induction  hypothesis 

T  b  Ci  :  A 

by  inversion  lemma 

□ 


The  proofs  are  straightforward  by  hand.  We  distinguish  cases  on  the  derivations  (or 
on  the  type),  reason  by  induction  hypothesis,  and  apply  lemmas  or  rules  of  inference 
to  deduce  the  correct  conclusion.  On  the  other  hand,  these  proofs,  in  particular  the 
final  type  preservation  theorem,  are  quite  challenging  for  an  automated  meta-theorem 
prover. 
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7.2  Twelf:  meta-theorem  prover 

In  previous  chapters,  we  outlined  how  to  implement  the  previous  type-system  in  Twelf 
as  a  higher-order  logic  program  and  discussed  how  to  execute.  In  this  chapter,  we  are 
interested  in  automating  the  reasoning  with  and  about  deductive  systems.  Examples  of 
such  meta-reasoning  are  the  admissibility  lemmas  and  the  equivalence  theorem  about 
the  refinement  type  systems  in  the  previous  section.  We  also  refer  to  these  properties 
as  meta-theorems. 

Schurmann  showed  in  his  Ph.D.  thesis  [63]  that  it  is  possible  to  reconcile  higher- 
order  abstract  syntax  and  inductive  reasoning  in  theory  and  even  automate  the  induc¬ 
tive  reasoning  in  practice.  The  theoretical  development  is  complex,  and  we  refer  the 
interested  reader  to  [63,  64],  In  this  section,  we  review  the  design  and  implementation 
of  the  inductive  meta-theorem  prover.  Closely  following  the  informal  proof  above,  the 
meta-theorem  prover  has  three  main  operations  it  iterates  over.  Case  analysis  (reason¬ 
ing  by  inversion),  computing  instances  of  the  induction  hypothesis,  proof  search  where 
we  directly  prove  the  current  conclusion  from  the  current  proof  assumptions  and  the 
rules  of  inference.  Proof  assumptions  are  either  instances  of  the  induction  hypothe¬ 
sis  or  other  assumptions  generated  applying  case  analysis,  and  assumptions  from  the 
statement  of  the  theorem.  Note  that  generating  instances  of  the  induction  hypothesis 
is  separated  from  proof  search  itself  and  added  to  the  set  of  proof  assumptions.  During 
proof  search,  induction  hypotheses  are  treated  as  any  other  proof  assumptions. 

The  design  reflects  the  three  key  operations  in  the  informal  proof:  analyzing  cases, 
generating  instances  of  the  induction  hypothesis,  and  deriving  a  proof  for  using  infer¬ 
ence  rules  and  proof  assumptions.  The  overall  meta-theorem  prover  simply  iterates 
over  all  the  three  operations,  until  a  proof  for  all  open  cases  has  been  found.  It  is 
important  to  point  out,  that  the  meta-theorem  prover  can  only  make  progress  in  the 
proof,  once  proof  search  has  failed  to  show  that  the  conclusion  is  derivable  from  the  cur¬ 
rent  proof  assumptions.  This  is  the  pre-requisite  to  splitting  a  conjecture  into  different 
cases.  Hence  quick  failure  is  essential  to  the  overall  performance  of  the  meta-theorem 
prover. 

Consider  the  proof  for  type  preservation  given  earlier.  First  the  meta-theorem 
prover  will  try  to  show  that  T  :  T  b  v  :  a  is  derivable  from  the  rules  of  inference,  the 
lemmas  and  the  assumptions  T>  :  T  b  m  :  a  and  £  :  m  v.  We  write  small  letters  v,  m, 
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Proof  search  based  on  logic  prograr 

program  clauses 
lemmas 

proof  assumptions 
induction  hypotheses 


Case  Splitting 

proof  assumptions 

induction  hypothesis 

Induction  hypothesis  general 

depth 


Figure  7.1:  Design  of  the  induction  meta-theorem  prover 


a  to  indicate  these  variables  are  universal  and  are  treated  like  constants.  If  direct  proof 
search  fails,  then  we  split  the  conjecture  into  different  cases.  There  are  two  possibilities: 
case  analysis  on  £  :  m  c— »  v  and  T>  :  T  h  m  :  a.  Using  heuristics,  the  meta-theorem 
prover  chooses  to  split  on  the  typing  derivation  T>  :  T  b  m  :  a.  Applying  inversion  on 
the  subtyping  rule,  this  leads  to  the  new  proof  assumptions  Y  b  m  :  a'  and  a'  <  a. 
Next,  we  generate  r  b  v  :  a'  which  is  an  instance  of  the  induction  hypothesis.  Finally, 
using  all  these  assumptions,  we  search  for  a  proof  of  Y  b  v  :  a.  We  can  establish  a 
proof  for  it,  but  using  the  proof  assumption  a'  <  a,  the  induction  hypothesis  T  b  v  :  a' 
and  the  subtyping  rule. 

To  complete  the  proof,  we  need  to  consider  the  remaining  12  cases  for  typing  rules. 

As  the  example  illustrate,  generic  proof  search  plays  an  important  role  in  establish¬ 
ing  a  proof  for  type  preservation  automatically.  It  is  first  used  to  show  that  no  proof 
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exists  for  the  original  statement  F  :  T  b  v  :  a,  which  is  a  prerequisite  for  applying 
case-analysis.  Second,  it  is  used  to  close  the  proof  in  each  of  the  cases  after  induction 
hypotheses  are  generated. 

Schiirmann  adopted  in  his  Ph.D.  thesis  the  simplest  solution  possible  for  searching 
for  a  proof  given  a  set  of  assumptions.  In  principle,  we  can  just  use  the  proof  search  pro¬ 
cedure  of  the  logic  programming  interpreter  to  search  for  a  proof.  The  main  drawback 
of  the  depth-first  search  strategy  used  in  the  logic  programming  is  that  it  is  incomplete 
and  may  get  stuck  in  a  branch,  and  never  consider  other  more  fruitful  branches  in 
the  proof  tree.  Therefore,  the  approach  taken  in  [63]  is  to  replace  the  incomplete  and 
unfair  depth-first  proof  search  strategy  of  the  logic  programming  interpreter  with  the 
fair  bounded  iterative  deepening  proof  search.  This  means  proof  search  will  still  be 
incomplete,  but  at  least  we  can  switch  to  other  potentially  more  fruitful  branches  in 
the  proof  tree,  once  we  have  reached  the  bound.  Although  this  seems  to  work  well 
in  many  examples,  it  has  several  shortcomings:  First,  the  user  has  to  specify  a  depth 
bound.  This  has  two  consequences.  If  a  proof  does  not  exist  within  the  given  bound, 
then  the  user  does  not  know  whether  the  stated  conjecture  is  improvable  or  the  bound 
was  set  to  low.  As  a  result,  failure  is  not  meaningful  in  this  setting  making  the  task  of 
developing  formal  systems  and  proofs  about  them  frustrating.  Second,  there  may  be 
redundancy  in  the  proof  search  hampering  the  overall  performance.  Note  that  quick 
failure  is  essential  in  the  overall  design  of  the  meta-theorem  prover,  to  make  progress 
and  split  the  conjecture  in  different  cases.  Let  us  briefly  consider  the  previous  example 
for  type  preservation  proof  again.  Before  we  can  split  the  conjecture  into  different 
cases,  we  need  to  show  that  T  :  T  h  v  :  a  is  not  derivable  from  the  current  proof 
assumptions  V  :  T  h  m  :  a  and  £  :  m  v,  inference  rules  and  lemmas.  One  of  the 
inversion  lemmas  and  the  subtyping  rule  are  applicable  and  can  be  used  in  the  first 
step.  This  can  quickly  lead  to  an  explosion  in  the  search  space.  In  practice  this  means 
iterative  deepening  gets  bogged  down  in  the  very  first  step  and  cannot  make  progress, 
thereby  failing  to  establish  an  inductive  proof  for  type  preservation. 

Third,  iterative  deepening  search  can  be  quite  fragile.  The  order  in  which  the 
inference  rules  are  implemented  matters  a  lot  and  proving  lemmas  and  adding  them  to 
the  database  of  known  facts  may  have  drastic  effects  on  how  well  proof  search  performs. 
While  it  is  reasonable  to  ask  the  user  to  carefully  implement  the  typing  rules,  the  user 
has  little  control  over  the  use  of  lemmas  and  other  proof  assumptions. 
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Iii  this  chapter  we  propose  memoization-based  search  as  an  alternative  to  iterative 
deepening.  Memoization-based  search  terminates  for  programs  over  a  finite  domain. 
For  those  programs  where  memoization-based  search  may  not  terminate,  the  user  still 
has  to  give  a  bound,  within  which  the  memoization-based  search  procedure  will  try 
to  find  a  proof  or  establish  that  no  proof  exists.  Experiments  show  that  handling 
redundancy  is  critical  to  prove  meta-theorems  like  the  ones  above  leading  to  a  more 
efficient  and  more  powerful  meta-theorem  prover. 


7.3  Generic  proof  search  based  on  memoization 

Memoization-based  proof  search  is  based  on  the  tabled  uniform  proof  system  given 
in  Chapter  4.  Although  the  underlying  ideas  and  large  parts  of  the  implementation 
are  shared  between  tabled  higher-order  logic  interpreter  and  the  memoization-based 
higher-order  proof  search,  there  are  a  few  key  difference:  1)  In  logic  programming, 
it  is  important  to  maintain  an  easy  to  understand  and  clearly  defined  operational 
semantics.  The  programmer  can  then  exploit  this  semantics  when  writing  programs. 
For  this  reason,  the  programmer  may  find  an  unfair  but  easy  to  understand  strategy 
preferable  in  logic  programming.  In  theorem  proving,  we  want  to  know  if  a  proof  exists 
and  an  unfair  search  strategy  is  unacceptable,  since  it  may  prevent  us  from  finding 
a  proof.  Since  memoization-based  proof  search  does  not  terminate  in  all  cases,  we 
cannot  eliminate  bounds  completely.  The  search  procedure  will  let  the  user  know,  if 
the  search  was  not  completed  within  this  bound.  However,  as  we  will  show  in  many 
cases  the  search  is  complete  thereby  providing  valuable  feedback  for  the  user  and  quick 
failure.  2)  In  logic  programming  we  have  a  fixed  set  of  program  clauses,  over  which 
the  programmer  has  control.  On  the  other  hand  in  meta-theorem  proving,  we  have 
additional  proof  assumptions  such  as  induction  hypotheses  and  lemmas,  which  are 
generated  during  proof  search,  and  over  which  the  programmer  has  no  control.  This 
makes  the  problem  of  search  for  a  proof  harder. 

Overall,  we  have  taken  a  very  conservative  approach  of  adding  memoization  to  the 
meta-theorem  prover.  We  only  replace  iterative  deepening  search  procedure  with  the 
new  memoization-based  search.  However  after  the  memoization-based  search  fails  or 
succeeds,  we  clear  the  memo-table.  A  more  aggressive  approach  could  be  to  have  the 
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memo-table  survive  across  iterations.  We  mainly  focus  on  using  variance  checking  in 
the  memoization-based  search.  The  main  reason  has  been  to  minimize  the  overhead  of 
maintaining  two  very  different  implementations,  one  for  the  interpreter  and  one  for  the 
meta-theorem  prover.  However,  we  will  show  using  proofs  about  refinement  types  that 
unlike  higher-order  logic  programming  the  use  of  subsumption  can  be  very  valuable  in 
meta-theorem  proving. 

We  also  discuss  some  of  differences  of  iterative  deepening  search  and  memoization- 
based  search.  (1)  Iterative  deepening  means,  we  first  try  to  find  a  proof  for  depth  1 
with  depth-first  search,  then  we  try  to  find  a  proof  of  depth  2  with  depth-first  search 
etc.  If  the  branch  is  deeper  than  k,  then  we  fail  and  backtrack.  This  means  if  there 
exists  a  proof  with  depth  k,  we  may  have  explored  the  left-most  branches  in  the  search 
tree  up  to  depth  k.  Obviously,  if  proof  search  fails,  then  we  have  to  explore  the  whole 
search  space.  (2)  Memoization-based  search  bounded  by  n:  we  traverse  the  search 
space  using  depth-first  search  and  memoization.  if  we  explore  a  branch  with  depth 
greater  than  n  we  fail  and  backtrack.  This  means  although  we  only  needed  depth  k, 
where  k  is  smaller  than  n,  we  explored  fruitless  branches  up  to  depth  n.  As  a  result, 
it  potentially  may  take  longer  to  find  solutions  to  some  problems. 

7.4  Experiments 

We  have  carried  out  several  experiments  so  far,  including  a  refinement  type  system 
(soundness  and  completeness  of  algorithmic  subtyping  algorithm),  type-system  with 
subtyping  (soundness  and  completeness  of  algorithmic  subtyping  and  type  preserva¬ 
tion)  and  theorem  proving  in  classical  natural  deduction.  The  meta-theorems  were  not 
previously  provable  with  the  iterative  deepening  proof  search,  thereby  demonstrating 
that  memoization-based  search  leads  to  a  more  powerful  meta-theorem  prover.  In  ad¬ 
dition,  we  give  some  examples  which  were  provable  with  iterative  deepening  with  a 
bound,  and  can  be  proven  without  a  bound  using  memoization-based  search. 

7.4.1  Type  preservation  for  type  system  with  subtyping 

First,  we  consider  the  proofs  about  the  type-system  with  subtyping  given  earlier.  The 
proofs  themselves  are  not  hard,  but  the  number  of  applicable  lemmas  increases  the 
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search  space  as  we  continue.  Lemmas  are  turned  into  logic  programming  clauses  which 
can  be  used  during  proof  search.  For  example,  the  lemma 

If  T  b  s  M  :  A  and  A  <j  pos  then  T  \-  M  :  nat. 

will  be  turned  into  the  clause 

invNat  :  of  M  nat 

<-  of  (s  M)  A 
<-  A  < |  pos 

From  a  logic  programming  view,  this  clause  may  lead  to  an  explosion  in  search  space. 
Therefore  we  see  that  iterative  deepening  proof  search  deteriorates  if  we  continue  and 
prove  more  lemmas  about  the  type-system.  Figure  7.2  gives  the  run-time  for  proving 
all  the  lemmas  and  theorems  needed  in  the  type  preservation  proof.  We  did  not  include 
any  speed-ups  in  this  table  since  many  numbers  are  very  small  and  for  those  the  speed¬ 
up  is  not  very  meaningful. 


theorem 

tabled 

iterative  deep 

refl 

0.18 

0.00 

trans 

0.29 

0.02 

sound 

0.19 

0.01 

comp 

0.21 

0.02 

imposAll 

0.04 

0.00 

inv+z+pos 

0.08 

0.00 

inv+suc+pos 

0.08 

0.00 

inv+suc+nat 

0.09 

0.23 

inv+suc+zero 

0.10 

0.93 

inv+lam 

0.21 

7.27 

inv+pair 

0.18 

oo 

tps 

2.89 

- 

Figure  7.2:  Type  preservation  for  system  with  snbtyping 


The  memoization-based  meta-theorem  prover  has  some  overhead  in  managing  the 
table  which  leads  to  slight  deterioration  in  the  first  few  examples.  However,  in  the  inver¬ 
sion  lemma  for  functions  memoization  starts  paying  off  and  the  proof  for  the  inversion 
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lemma  for  pairs  cannot  be  found  by  iterative  deepening  search  within  a  reasonable 
amount  of  time.  The  main  benefits  of  memoization  are  not  in  quickly  hireling  a  proof 
for  a  sub-case  in  the  proof,  but  rather  in  quickly  failing,  i.e.  in  showing  no  proof  exists 
from  the  current  proof  assumptions,  and  therefore  triggering  a  case  split.  The  proof  of 
the  inversion  lemma  for  functions  for  example  has  between  37  and  67  suspended  goals. 
The  proof  of  type  preservation  has  between  31  and  over  70  suspended  goals.  All  these 
loops  were  detected  when  proving  that  it  is  not  possible  to  derive  the  conclusion  from 
the  current  set  of  assumptions.  Hence,  the  overall  performance  improvement  is  mainly 
due  to  quick  failure  which  allows  the  theorem  prover  to  make  progress.  As  a  result,  we 
succeed  in  proving  all  the  lemmas  including  the  final  type  preservation  theorem  using 
the  memoization-based  prover.  The  iterative  deepening  prover  gets  stuck  during  the 
proof  for  the  inversion  lemma  for  pairs.  Even  if  we  add  the  inversion  lemma  for  pairs 
without  a  proof  to  the  set  of  lemmas,  the  iterative  deepening  prover  does  not  succeed 
in  proving  type  preservation. 


7.4.2  Derived  rules  of  inference  in  classical  natural  deduction 

Next,  we  show  some  experiments  using  classical  natural  deduction.  Proof  search  in 
natural  deduction  calculi  is  considered  hard.  Memoization  can  lead  to  quicker  proofs 
in  some  examples,  while  imposing  a  some  performance  penalty  in  others.  In  Table  7.3, 
we  give  the  run-times  for  proving  several  derived  rules.  These  examples  do  not  require 
induction. 


theorem 

tabled  search 

depth- first-bounded 

iterative  deepening 

split’ 

9.20 

0.19 

0.29 

join’ 

9.18 

1.68 

1.69 

assoc 

20.14 

450.53 

451.49 

uncurry 

33.45 

450.97 

452.42 

Figure  7.3:  Derived  rules  of  inference  in  NK  (classical  natural  deduction  with  not,  imp, 
and,  double  negation  rule) 
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7.4.3  Refinement  types 

In  this  sections  we  used  the  memoization-based  prover  to  proof  all  the  lemmas  leading 
up  to  the  type  preservation  theorem.  We  use  an  implementation  of  the  type  system  for 
refinement  types  by  Pfenning  [17].  For  proving  these  statements,  we  used  subsumption- 
based  memoization.  It  is  important  to  note,  that  this  example  clearly  demonstrates  the 
use  of  subsumption,  since  with  variant-based  memoization  we  still  cannot  find  some 
proofs.  Subsumption  is  critical  to  detect  quickly  that  no  proof  exists  and  we  need 
to  split  the  conjecture  into  different  cases.  It  also  is  critical  to  complete  some  of  the 
sub-proofs2. 

Iterative  deepening  works  fine  for  the  first  few  proofs.  In  fact,  its  performance  is 
better  than  memoization-based  proof  search.  However,  it  cannot  prove  the  inversion 
lemmas.  Attempting  the  proof  for  the  inversion  lemma  for  functions,  iterative  deep¬ 
ening  cannot  find  a  proof  within  a  reasonable  amount  of  time.  When  skipping  the 
inversion  lemma  for  functions,  we  are  able  to  make  some  more  progress  by  proving  the 
two  impossibility  lemma,  but  we  get  again  stuck  when  proving  the  lemma  imp+l+zero. 
Some  of  the  lemmas,  we  are  able  to  prove  individually.  However,  this  is  unsatisfactory 
for  developing  the  meta-proofs  about  the  refinement  types. 

Using  subsumption-based  memoization,  we  detected  over  100  loops  when  proving 
inversion  lemmas.  Similar  to  the  subtyping  example,  memoization  was  most  valuable 
for  detecting  that  the  current  proof  assumptions  do  not  suffice  to  derive  the  conclusion. 

7.4.4  Rewriting  semantics 

Next,  we  give  two  examples  which  were  provable  before  but  did  require  a  bound.  In 
fact,  the  user  has  to  choose  the  minimal  bound  in  order  to  get  the  meta-theorem 
prover  to  prove  these  theorems.  With  memoization,  no  bounds  are  required.  This  is 
especially  important  if  we  want  to  disprove  conjectures  and  debug  implementations 
and  proofs  about  them.  We  first  consider  a  rewriting  semantic  with  weak  congruence 
closure  and  show  that  any  big-step  evaluation  has  a  sequence  of  rewriting  steps  in 
the  reduction  semantic  (evalRed  theorem).  Next,  we  try  to  prove  that  any  rewriting 
sequence  has  a  corresponding  big-step  evaluation  (RedEval  theorem).  However,  this 

2run-times  for  variant-based  memoization  to  be  added 
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theorem 

tabled  search  (subsumption) 

iterative  deepening 

lemmal 

3.72 

0.08 

refl 

0.21 

0.00 

trans 

1.63 

0.02 

compj— 

1.81 

0.01 

sound  j— 

0.41 

0.01 

inv+lam 

0.36 

- 

impossAll 

0.020 

- 

irnpRV 

0.140 

- 

imp+e+pos 

0.220 

- 

imp+l+zero 

0.260 

- 

imp+O+zero 

0.28 

- 

inv+e+bits 

0.080 

- 

inv+e+nat 

0.08 

- 

inv+l+bits 

0.450 

- 

inv+O+bits 

0.500 

- 

inv+O+nat 

0.520 

- 

inv+l+nat 

0.510 

- 

inv+O+pos 

0.510 

- 

inv+l+pos 

0.490 

- 

co  nj  Lemma 

0.110 

- 

Figure  7.4:  Refinement  type  checking  [17] 


217 


CHAPTER  7.  HIGHER-ORDER  THEOREM  PROVING  WITH  TABLING 


is  in  fact  not  true  the  way  the  rewrite  rules  are  stated,  since  we  not  always  will  get 
back  a  value.  Memoization-based  search  will  return  and  say  that  no  proof  is  possible. 
This  in  turn  is  valuable  to  analyze  the  specification  and  potentially  change  it.  Finally, 
we  include  a  type  preservation  theorem  about  the  reduction  semantics.  Run-times  for 
the  memoization-based  prover  are  slightly  worse  than  the  iterative  deepening  prover. 
However,  we  get  a  more  expressive  prover  which  doesn’t  require  bounds. 


theorem 

tabled  search 

iterative  deepening 

evalRed  (sound) 

0.37sec 

O.Olsec 

RedEval  (complete) 

9.88sec 

- 

typeRed 

0.93sec 

O.Olsec 

Figure  7.5:  Proofs  about  the  rewrite  semantics  vs  big-step  evaluation  semantics 

Similarly,  for  the  proofs  about  Mini-ml  we  observe  that  the  theorems  fall  into  the 
complete  fragment,  and  the  theorem  prover  does  not  require  any  bounds.  Again  we 
include  the  proofs  about  the  original  specification  and  contrast  it  with  slightly  modified 
specification  where  the  proofs  about  type  preservation  and  value  soundness  fail. 


theorem 

tabled  search 

iterative  deepening 

val-sound 

0.33sec 

0.32sec 

tp-preseve 

0.53sec 

O.Olsec 

tp-preserve  (mod) 

1.18sec 

- 

Figure  7.6:  Proofs  about  big-step  evaluation  semantics,  type  preservation,  value  sound¬ 
ness,  mini-ml 


7.5  Conclusion 

Redundancy  elimination  is  critical  to  reason  with  and  about  non-trivial  deductive  sys¬ 
tems.  We  have  shown  that  memoization-based  search  can  be  smoothly  integrated  into 
the  meta-theorem  prover.  This  allows  us  to  prove  more  complicated  theorems  which 
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rely  on  several  lemmas.  From  our  experience,  the  better  failure  behavior  is  particularly 
useful  in  developing  and  debugging  specifications  and  theorems.  In  addition,  it  leads 
to  a  substantial  improvement  of  the  overall  performance  of  the  meta-theorem  prover. 
As  we  are  able  to  fail  quicker,  we  are  able  to  make  progress  faster. 

However,  the  conservative  memoization-based  search  also  has  some  limitations, 
which  we  plan  to  address  in  the  future.  In  particular  the  proofs  about  refinement 
types  illustrate  that  subsumption  based  memoization  is  useful  in  theorem  proving.  As 
we  mentioned  earlier,  we  have  only  an  incomplete  subsumption-based  memoization 
strategy  in  the  current  implementation,  to  minimize  the  effort  required  to  maintain 
different  implementations  for  variant-  and  subsumption-based  memoization.  In  the 
long  run,  it  seems  interesting  to  develop  a  complete  subsumption-based  memoization 
proof  search  procedure  for  the  meta-theorem  prover. 
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Chapter  8 
Conclusion 


In  this  thesis,  we  develop  different  techniques  to  efficiently  execute  and  reason  with  and 
about  deductive  systems.  We  have  shown  that  memoization  as  a  form  of  redundancy 
elimination  is  a  useful  extension  to  the  higher-order  logic  programing  interpreter.  It 
allows  us  to  execute  more  examples  and  have  better  and  more  meaningful  failure.  It 
also  plays  an  important  role  in  the  meta-theorem  prover.  To  achieve  good  performance 
of  memoization-based  search,  we  have  developed  higher-order  term  indexing  techniques 
and  an  optimized  higher-order  pattern  unification  algorithm,  which  eliminates  many 
unnecessary  occurs  checks.  Taken  together  the  presented  techniques  constitute  a  sig¬ 
nificant  step  toward  exploring  the  full  potential  of  logical  frameworks  in  real-world 
applications  where  deductive  systems  and  proofs  about  them  may  be  more  complex. 


Future  Work 

Applications:  Typed  Assembly  Language 

Recently,  Twelf  has  been  used  in  developing  a  foundational  approach  for  typed  as¬ 
sembly  language  [14,  15].  This  approach  is  similar  to  the  foundational  proof-carrying 
code  project  [2]  in  the  sense  that  we  try  to  minimize  the  trusted  computing  base.  In 
particular,  we  do  not  need  to  trust  the  safety  policy,  but  the  safety  policy  is  proven 
sound.  While  in  the  foundational  proof-carrying  code  approach  higher-order  logic  to¬ 
gether  with  some  axioms  about  arithmetic  is  used  as  a  foundation  to  define  safety 
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policies,  the  logical  framework  LF  itself  serves  as  the  foundation  for  implementing  dif¬ 
ferent  safety  policies.  In  addition  to  the  safety  policy,  described  by  a  type  system, 
all  the  meta-proofs  required  to  prove  soundness  are  implemented  in  Twelf.  However, 
so  far  Twelf  has  been  mainly  used  as  a  specification  framework  and  not  fully  uti¬ 
lized  the  expressiveness  and  power  of  execution  and  reasoning  strategies.  There  are 
many  applications  of  the  described  work  to  the  foundational  typed  assembly  language 
project.  First,  it  seems  interesting  to  experiment  and  generate  safety  certificates  for 
sample  programs  using  Twelf’s  logic  programming  interpreter.  Second,  it  would  be 
interesting  to  investigate  if  some  of  the  meta-theorems  about  the  safety  policy  can  be 
proven  automatically  using  the  memoization-based  meta-theorem  prover.  Third,  we 
can  instrument  Twelf’s  higher-order  logic  programming  interpreter  to  produce  a  bit¬ 
string  that  encodes  the  non-deterministic  choices  in  a  proof  and  use  memoization  to 
factor  out  common  subproofs.  This  extends  prior  work  by  Necula  and  Rahul  [42]  to 
full  LF.  Moreover,  preliminary  experiments  demonstrate  that  memoization  can  lead  to 
bit-string  which  are  up  to  40%  smaller  in  size  than  their  counterparts  produced  without 
memoization. 

Eliminating  further  redundant  typing  information 

There  is  still  some  redundancy  in  proof  search  which  is  due  to  dependent  types.  De¬ 
pendent  type  information  needs  to  be  carried  around  during  proof  search  imposing  a 
substantial  overhead  [33].  Necula  and  Lee  [43]  have  investigated  the  problem  of  redun¬ 
dant  type  information  in  the  context  of  type-checking.  They  propose  a  more  efficient 
term  representation  for  a  fragment  of  LF  which  eliminates  a  lot  of  the  redundant  typing 
information  thereby  minimizing  the  amount  of  information  we  need  to  keep  for  type¬ 
checking.  One  direction  is  to  extend  Necula  and  Lee’s  work  to  full  LF  type  theory, 
thereby  eliminating  further  redundancy.  Moreover,  it  may  be  used  to  not  only  improve 
the  performance  of  type-checking  in  Twelf,  but  also  of  proof  search  itself. 

Completeness  of  memoization-based  proof  search 

In  this  thesis,  we  have  mainly  focused  on  proving  soundness  of  memoization-based  proof 
search.  Completeness  of  memoization-based  proof  search  is  harder  to  establish.  The 
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main  reason  is  that  we  will  find  fewer  proofs  than  with  uniform  proofs.  For  example 
in  the  subtyping  example  given  in  Sec.  4.1  the  query  sub  zero  zero  has  infinitely  many 
proofs  under  the  traditional  logic  programming  interpretation  while  we  find  only  one 
proof  under  the  tabled  logic  programming  interpretation.  However,  we  often  do  not 
want  or  need  to  distinguish  between  different  proofs  for  a  formula  A,  but  only  care  about 
the  existence  of  a  proof  for  A  together  with  a  proof  term.  In  [50]  Pfenning  develops 
a  dependent  type  theory  for  proof  irrelevance  and  discusses  potential  applications  in 
the  logical  framework.  This  could  allow  us  to  treat  all  proofs  for  A  as  equal  if  they 
produce  the  same  answer  substitution.  In  this  setting,  it  seems  possible  to  show  that 
search  based  on  tabled  uniform  proofs  is  complete,  i.e.  if  there  is  a  (canonical)  proof 
M  :  A  then  we  can  find  A. 

Optimizations  for  memoization-based  search 

Re-use  of  answers 

Memoization-based  search  is  critically  influenced  by  when  we  retrieve  answers  and  when 
we  suspend  goals  and  when  and  in  what  order  we  awaken  suspended  goals.  Currently, 
we  have  implemented  multi-stage  depth-first  strategy.  As  we  have  discussed  in  Chapter 
4,  this  strategy  might  delay  the  re-use  of  answers,  since  we  are  not  allowed  to  retrieve 
answers  from  the  current  stage.  The  XSB  system  therefore  uses  a  strategy  based  on 
strongly  connected  components,  which  allows  us  to  consume  answers  as  soon  as  they  are 
available.  We  build  a  dependency  graph  of  the  predicates  and  identify  different  inde¬ 
pendent  sub-graphs,  called  strongly  connected  components.  This  dependency  analysis 
separates  subgoals  that  can  and  cannot  influence  each  other.  If  computation  for  all 
the  subgoals  within  a  sub-graph  is  saturated,  then  we  can  dispose  all  the  suspended 
sub-goal  belonging  to  this  component.  As  we  are  able  to  delete  suspended  nodes  that 
cannot  contribute  to  new  solutions  anymore,  this  leads  to  fewer  suspended  nodes  and 
has  the  potential  to  be  more  space  and  time  efficient.  This  might  be  advantageous  es¬ 
pecially  in  theorem  proving,  where  we  only  care  about  one  answer  to  the  query  and  are 
not  interested  in  mimicking  Prolog  execution.  In  LF,  subordination  analysis  provides 
information  about  dependencies  of  predicates  and  constructs  a  subordination  graph. 
Analyzing  and  exploiting  the  subordination  information,  it  seems  possible  to  design 


223 


CHAPTER  8.  CONCLUSION 


similar  strategies  as  in  XSB  which  allow  the  re-use  of  answers  as  soon  as  they  are 
available  and  detect  when  all  answers  have  been  generated. 

Subsumption 

As  some  of  the  experiments  in  Chapter  7  suggest,  the  use  of  subsumption-based  mem- 
oization  in  proof  search  is  useful  in  meta-theorem  proving.  As  we  have  also  mentioned 
in  Chapter  7,  so  far  the  implementation  of  subsumption  is  a  conservative  extension  of 
the  variant-based  memoization  proof  search  and  may  be  incomplete.  In  the  future,  it 
seems  fruitful  to  develop  a  complete  subsumption-based  proof  search  procedure  for  the 
meta-theorem  prover. 

High-level  optimizations  to  proof  search 

Although  memoization-based  computation  aims  to  make  reasoning  within  logical  speci¬ 
fication  efficient,  it  cannot  rival  the  performance  of  a  theorem  prover  that  is  specifically 
built  for  a  given  logic.  One  reason  is  that  specialized  state-of-the-art  theorem  provers 
exploit  properties  of  the  theory  they  are  built  for.  For  example,  inverse  method  theorem 
provers  for  first-order  logic  like  Gandalf  [66]  exploit  the  subformula  property.  Other 
theorem  provers  like  Spass  [69] ,  which  are  very  successful  in  equational  reasoning,  rely 
on  orderings  to  restrict  the  search.  These  meta-level  optimizations  can  improve  per¬ 
formance  of  higher-order  logic  programming  dramatically.  Therefore,  one  interesting 
path  to  explore  is  to  verify  such  properties  about  logical  specifications  in  advance  and 
exploit  them  during  search. 

Modal  dependent  type  theory 

In  Chapter  2,  we  have  conservatively  extended  the  LF  type  theory  with  modal  variables 
and  shown  that  canonical  forms  exist  and  type-checking  remains  decidable.  As  we 
have  briefly  discussed  in  Chapter  2,  there  are  two  simple  and  clean  ways  to  incorporate 
such  variables.  One  is  via  a  general  modal  operator  and  the  other  is  via  a  new 
quantifier  IIDu::('Fi-Ai).  Ai.  The  main  complication  with  the  general  modal  operator 
is  that  canonical  forms  do  not  exist.  In  this  thesis,  we  suggested  the  use  of  IID  to 
quantify  over  modal  variables  explicitely,  but  we  have  not  fully  extended  the  LF  type 
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theory.  In  the  future,  it  seems  interesting  to  develop  the  theory  further  towards  a  modal 
dependent  type  theory.  The  IP-abstraction  and  the  □-application  is  less  powerful  than 
the  general  box-operator.  We  conjecture  that  canonical  forms  exist  for  this  fragment 
and  type-checking  remains  decidable. 

In  Chapter  3,  we  have  briefly  mentioned  an  extension  which  allows  us  to  quantify 
directly  over  variable  definitions  using  IIDw  =  This  would  have  two  main 

advantages:  First,  we  would  be  able  to  type-check  linearized  objects  L  directly.  This 
avoids  translating  variable  definitions  into  a  modal  substitution  9  and  the  type-checking 
\6\L.  Second,  it  would  allow  us  to  describe  proof  terms  as  directed-acyclic-  graph  and 
type-check  them  directly. 

Linear  logic  programming 

Linear  logic  programming  [30,  8]  has  been  proposed  as  an  extension  of  higher-order 
logic  programming  to  model  imperative  state  changes  in  a  declarative  (logical)  way.  We 
believe  that  the  some  of  the  techniques  presented  in  this  thesis  can  be  extended  to  linear 
logic  programming,  but  it  requires  some  new  considerations.  In  particular,  we  plan  to 
investigate  the  use  of  memoization  in  linear  higher-order  logic  programming.  This 
requires  new  consdiderations  concerning  the  interaction  between  resource  management 
strategies  [10]  or  constraints  [27]  with  tabling.  However,  the  approach  presented  seems 
general  and  amenable  also  to  the  linear  case. 
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